ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
449 commits 391 branches 58 tags 304 MiB
Commit graph

449 commits

This branch
This branch
All branches
Author SHA1 Message Date
Daniel Han
e5381b6a26 Update _utils.py 2024-09-22 02:38:28 -07:00
Nazim Ali
02cc97e388
 		fix: chat_templates.py bug (#1048) 
* fix: chat_template bug

* fix: check trainer attribute values are not None
 2024-09-22 01:18:37 -07:00
Daniel Han
c247bfc72c Update chat_templates.py 2024-09-21 01:56:17 -07:00
Daniel Han
0fbbdfc091 Merge branch 'nightly' 2024-09-18 14:30:33 -07:00
Daniel Han
3fddfd5166 Update mapper.py 2024-09-18 14:23:22 -07:00
Daniel Han
8aceff3e7b
 		Update README.md (#1036) 2024-09-18 13:23:45 -07:00
Daniel Han
c730659de7 Update llama.py 2024-09-17 17:38:12 -07:00
Daniel Han
f1951c0f6d Update mapper.py 2024-09-17 10:50:57 -07:00
Daniel Han
62c989ef0a Update mapper.py 2024-09-15 21:50:00 -07:00
Daniel Han
572c925fa7 Update _utils.py 2024-09-15 18:04:18 -07:00
Daniel Han
575c1bd67d
 		Update README.md (#1033) 2024-09-15 17:42:09 -07:00
Daniel Han
6c534341bb Update utils.py 2024-09-08 19:47:21 -07:00
Daniel Han
de43b9cedc Update __init__.py 2024-09-08 15:51:27 -07:00
Daniel Han
7476d4b5f6 Update README.md 2024-09-08 14:30:54 -07:00
Daniel Han
d674f1c852 Update README.md 2024-09-08 12:29:31 -07:00
Daniel Han
f549a5473c
 		Bug fixes (#1004) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update token retrieval logic (#952)

* Fix DPO (#947)

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update hf token retrieval logic

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update llama.py

* get_token

* Update README.md

* Update gemma2.py

* Update rms_layernorm.py

* synchronize

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* layernorm

* Update rms_layernorm.py

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* revert

* Gemma

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update gemma2.py

* Change UnslothTrainingArguments base class to SFTConfig (#979)

* Cohere

* Update trainer.py

* Cohere

* Cohere

* New models

* Update llama.py

* Update llama.py

* Update cohere.py

* Update llama.py

* Update cohere.py

* retry

* Update fast_lora.py

* Update llama.py

* Update fast_lora.py

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* _apply_lora_mlp

* Update _utils.py

* Gemma fixes

* Update llama.py

* Update flex_attention.py

* Update llama.py

* layernorm

* Update llama.py

* Update llama.py

* Flex Attention

* Update gemma2.py

* Update __init__.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update chat_templates.py (#999)

fix all misspelled "unsued" to "unused"

* Update key from "from" to "user" (#1000)

When use [tokenizer.apply_chat_template](https://huggingface.co/docs/transformers/main/en/chat_templating), the key should be "role" rather than "from", this is liknk to [this issue](https://github.com/unslothai/unsloth/issues/994)

I don't know it is suitable for all situation, I also can add a dedicated parameter of the key if you think it is better.

* Update chat_templates.py

* Also patch the KTO trainer (#1001)

* flex attention

* Update llama.py

* Update flex_attention.py

* Update flex_attention.py

* Update _utils.py

* Update _utils.py

* Update flex_attention.py

* Update gemma2.py

* Update gemma2.py

---------

Co-authored-by: Hafedh <70411813+not-lain@users.noreply.github.com>
Co-authored-by: Tuan Pham <82665400+vTuanpham@users.noreply.github.com>
Co-authored-by: Yihao Wang <42559837+AgainstEntropy@users.noreply.github.com>
Co-authored-by: Peng <zphu1024@gmail.com>
Co-authored-by: Kyle Corbitt <kyle@openpipe.ai>
 2024-09-08 03:16:09 -07:00
Daniel Han
d91d40a7b6 Bug fixes 2024-09-04 00:28:53 -07:00
Daniel Han
3682672d73 Fix bug 2024-09-03 17:30:40 -07:00
Daniel Han
480f2ef31c
 		Gemma faster inference (#987) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update token retrieval logic (#952)

* Fix DPO (#947)

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update hf token retrieval logic

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update llama.py

* get_token

* Update README.md

* Update gemma2.py

* Update rms_layernorm.py

* synchronize

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* layernorm

* Update rms_layernorm.py

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* revert

* Gemma

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update gemma2.py

* Change UnslothTrainingArguments base class to SFTConfig (#979)

* Cohere

* Update trainer.py

* Cohere

* Cohere

* New models

* Update llama.py

* Update llama.py

* Update cohere.py

* Update llama.py

* Update cohere.py

* retry

* Update fast_lora.py

* Update llama.py

* Update fast_lora.py

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* _apply_lora_mlp

* Update _utils.py

* Gemma fixes

* Update llama.py

* Update flex_attention.py

---------

Co-authored-by: Hafedh <70411813+not-lain@users.noreply.github.com>
Co-authored-by: Tuan Pham <82665400+vTuanpham@users.noreply.github.com>
 2024-09-03 13:52:12 -07:00
Daniel Han
c085a4562c
 		Cohere, Bug fixes (#984) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update token retrieval logic (#952)

* Fix DPO (#947)

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update hf token retrieval logic

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update llama.py

* get_token

* Update README.md

* Update gemma2.py

* Update rms_layernorm.py

* synchronize

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* layernorm

* Update rms_layernorm.py

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* revert

* Gemma

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update gemma2.py

* Change UnslothTrainingArguments base class to SFTConfig (#979)

* Cohere

* Update trainer.py

* Cohere

* Cohere

* New models

* Update llama.py

* Update llama.py

* Update cohere.py

* Update llama.py

* Update cohere.py

* retry

* Update fast_lora.py

* Update llama.py

* Update fast_lora.py

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* _apply_lora_mlp

* Update _utils.py

---------

Co-authored-by: Hafedh <70411813+not-lain@users.noreply.github.com>
Co-authored-by: Tuan Pham <82665400+vTuanpham@users.noreply.github.com>
 2024-09-03 01:52:32 -07:00
Daniel Han
976d11a10d Update save.py 2024-08-27 00:08:39 -07:00
Daniel Han
12b437e122 Update gemma2.py 2024-08-23 23:43:57 -07:00
Daniel Han
20506463e1
 		Phi 3.5 bug fix (#955) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update token retrieval logic (#952)

* Fix DPO (#947)

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update hf token retrieval logic

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update llama.py

* get_token

* Update README.md

---------

Co-authored-by: Hafedh <70411813+not-lain@users.noreply.github.com>
 2024-08-23 17:38:24 -07:00
Daniel Han
d0ca3497eb
 		Fix DPO (#947) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py
 2024-08-22 02:18:03 -07:00
Daniel Han
600ffe2a17
 		Update README.md (#941) 
Co-authored-by: Michael <107991372+shimmyshimmer@users.noreply.github.com>
 2024-08-20 17:59:50 -07:00
Daniel Han
f629a638d6 Update chat_templates.py 2024-08-20 16:54:11 -07:00
Daniel Han
d5768379c5
 		Phi 3.5 (#940) 
* LongRoPE

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update mapper.py

* Phi 3.5
 2024-08-20 16:51:39 -07:00
Daniel Han
d45ade257b
 		Update README.md (#938) 2024-08-19 17:18:30 -07:00
Daniel Han
8283cccf95 Merge branch 'main' into nightly 2024-08-19 17:13:12 -07:00
Daniel Han
fbf50a4260 Update _auto_install.py 2024-08-19 17:12:46 -07:00
Daniel Han
4cc20f4720 Create _auto_install.py 2024-08-19 17:12:32 -07:00
Daniel Han
75013ff022
 		Fix NEFTune (#937) 
* untrained tokens llama 3.1 base

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Bug fixes

* Update llama.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update llama.py

* Update llama.py

* Update llama.py
 2024-08-19 16:17:52 -07:00
Daniel Han
f65ead3e48 Merge branch 'main' into nightly 2024-08-19 16:17:00 -07:00
Daniel Han
be7ed9a1e6 Update llama.py 2024-08-19 16:14:01 -07:00
Daniel Han
9caaa5af78 Update llama.py 2024-08-19 16:11:09 -07:00
Daniel Han
7139e57b72 Update llama.py 2024-08-19 16:08:14 -07:00
Daniel Han
6e5ad15cd7 Update tokenizer_utils.py 2024-08-19 16:03:24 -07:00
Daniel Han
7fd058fc71 Update tokenizer_utils.py 2024-08-19 15:52:13 -07:00
Daniel Han
2c4772b666 Update tokenizer_utils.py 2024-08-19 15:50:16 -07:00
Daniel Han
1bed78c992 Update llama.py 2024-08-19 15:08:53 -07:00
Daniel Han
9335fa0960 Bug fixes 2024-08-19 15:04:25 -07:00
Daniel Han
52bc19d1fa
 		Bug #930 (#931) 
* untrained tokens llama 3.1 base

* Update tokenizer_utils.py

* Update tokenizer_utils.py
 2024-08-16 23:39:44 -07:00
Daniel Han
487637db7b Update tokenizer_utils.py 2024-08-16 23:38:43 -07:00
Daniel Han
9cb5c2eca4 Update tokenizer_utils.py 2024-08-16 23:38:02 -07:00
Daniel Han
0c5ba8dd18 Merge branch 'main' into nightly 2024-08-16 23:37:16 -07:00
Daniel Han
c22162b402
 		untrained tokens llama 3.1 base (#929) 2024-08-16 19:57:19 -07:00
Daniel Han
5e2cf1c51c untrained tokens llama 3.1 base 2024-08-16 19:28:43 -07:00
Daniel Han
8b80820b8b Update __init__.py 2024-08-15 15:07:42 -07:00
Daniel Han
8be73b1086 Bug fixes 2024-08-15 15:04:46 -07:00
Daniel Han
53cd1e7781
 		Fix mapping (#921) 
* Update pyproject.toml

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* fix_tokenizer

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update loader.py

* Update pyproject.toml

* Update _utils.py

* Update gemma2.py

* Update gemma2.py

* Update _utils.py

* gemma 2 mask

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update _utils.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update _utils.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Torch 2.4 Xformers 0.0.27post2

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Gemma 2 fixes

* Update gemma2.py

* Update llama.py

* Update llama.py

* Update save.py

* Update save.py

* Update llama.py

* Update cross_entropy_loss.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Update dpo.py

* Providing more flexibility for users to customize their llama when using LoRA (#910)

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update chat_templates.py

* return model

* Update tokenizer_utils.py

* Update chat_templates.py

* Update tokenizer_utils.py

* Train on completions

* load_in_4bit=False broken

* Update llama.py

* MAP_TO_UNSLOTH_16bit

* Update loader.py

* Update loader.py

* Update loader.py

* Update loader.py

* Update mapper.py

* Update mapper.py

* works!

---------

Co-authored-by: Po-Lung Wang <Brownwang0426@gmail.com>
 2024-08-15 01:15:35 -07:00