ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
549 commits 391 branches 58 tags 304 MiB
Commit graph

549 commits

This branch
This branch
All branches
Author SHA1 Message Date
Daniel Han
dfdff91258 Fix 4.47 issue (#1182) 
* Fix TRL

* Update mistral.py

* Patch processing_class

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Installation guide (#1165)

* chore: update chat_templates.py (#1166)

orginal -> original

* Disable Flex Attention

* Update tokenizer_utils.py

* Update _utils.py

* n_items

* Update cross_entropy_loss.py

* Fix DPO, ORPO

* Update _utils.py

* Update _utils.py

* fix/transformers-unpack (#1180)

* Fix DPO, ORPO (#1177)

* Fix TRL

* Update mistral.py

* Patch processing_class

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Installation guide (#1165)

* chore: update chat_templates.py (#1166)

orginal -> original

* Disable Flex Attention

* Update tokenizer_utils.py

* Update _utils.py

* n_items

* Update cross_entropy_loss.py

* Fix DPO, ORPO

* Update _utils.py

---------

Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>

* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>

* Update cross_entropy_loss.py

* Update _utils.py

* Update _utils.py

---------

Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
 2024-10-24 12:17:21 -07:00
Daniel Han
6a90ba64b7 Fix DPO, ORPO (#1177) 
* Fix TRL

* Update mistral.py

* Patch processing_class

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Installation guide (#1165)

* chore: update chat_templates.py (#1166)

orginal -> original

* Disable Flex Attention

* Update tokenizer_utils.py

* Update _utils.py

* n_items

* Update cross_entropy_loss.py

* Fix DPO, ORPO

* Update _utils.py

---------

Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
 2024-10-24 00:36:37 -07:00
Daniel Han
acef7d6a3d Update _utils.py 2024-10-23 12:39:58 -07:00
Edd
f402d945ff Fix/patch tokenizer (#1171) 
* fix: correct tokenizer handling in patch_sft_trainer_tokenizer

* Revert "fix: correct tokenizer handling in patch_sft_trainer_tokenizer"

This reverts commit 7a98e465cbd4f980c8b364b0396d44f2d052090f.

* fix: correct condition for test_text assignment in patch_sft_trainer_tokenizer
 2024-10-23 12:32:33 -07:00
Daniel Han
e5e67f83b4 Many bug fixes (#1162) 
* Fix TRL

* Update mistral.py

* Patch processing_class

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Installation guide (#1165)

* chore: update chat_templates.py (#1166)

orginal -> original

* Disable Flex Attention

* Update tokenizer_utils.py

* Update _utils.py

---------

Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
 2024-10-23 03:14:57 -07:00
Daniel Han
c3c9e1604b Update save.py 2024-10-20 01:52:21 -07:00
Daniel Han
251c3f9af2 Update _utils.py 2024-10-18 23:10:30 -07:00
Daniel Han
ac1c0fbaed Fix get_token 2024-10-18 23:08:30 -07:00
vo1d-ai
5dcc0cd613 fix: compute_loss bug (#1151) 
Currently, Unsloth doesn't pass additional parameters to Trainer.compute_loss such as return_outputs. This leads to errors when calling trainer.evaluate(). This change fixes the bug by properly passing parameters to Trainer.compute_loss.
 2024-10-18 20:46:07 -07:00
Daniel Han
94087e83d9 Update _utils.py 2024-10-17 20:50:05 -07:00
Daniel Han
f14e49d10f Update README.md 2024-10-17 20:46:11 -07:00
Daniel Han
a158a15965 Update README.md 2024-10-17 20:45:40 -07:00
Daniel Han
5e0a4bede8 Gradient Accumulation Fix (#1146) 
* Unsloth Zoo

* Update trainer.py

* Update trainer.py

* Update cross_entropy_loss.py

* n_items

* Update llama.py

* kwargs

* Remove extraneous f prefixes (#1133)

Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>

* Update __init__.py

* kwargs

* Update trainer.py

* Update trainer.py

* Update trainer.py

* Fix GA

* Update _utils.py

* Update llama.py

* Update tokenizer_utils.py

* Warn on old versions

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

---------

Co-authored-by: Emil Sadek <esadek@hotmail.com>
Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>
 2024-10-17 20:43:07 -07:00
Daniel Han
98d68dae2b Update mapper.py 2024-10-16 21:48:05 -07:00
Daniel Han
c05e87f18d Gradient Accumulation Fix (#1134) October-2024 
* Unsloth Zoo

* Update trainer.py

* Update trainer.py

* Update cross_entropy_loss.py

* n_items

* Update llama.py

* kwargs

* Remove extraneous f prefixes (#1133)

Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>

* Update __init__.py

---------

Co-authored-by: Emil Sadek <esadek@hotmail.com>
Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>
 2024-10-14 19:17:35 -07:00
Daniel Han
f75a92d14d Update save.py 2024-10-11 00:00:06 -07:00
Daniel Han
71ac721cec Update save.py 2024-10-10 23:22:17 -07:00
Giulia Baldini
4ed744ef16 Only remove folder in sentenpiece check if it was created (#1121) 2024-10-10 23:21:27 -07:00
Giulia Baldini
3d00fbd7c7 Handle absolute paths using pathlib (#1120) 2024-10-10 23:20:34 -07:00
Daniel Han
8100844172 Reload 2024-10-05 17:21:48 -07:00
Daniel Han
bf6a87a608 Merge branch 'nightly' 2024-10-01 00:45:16 -07:00
Daniel Han
98a4a570ad Update README.md 2024-10-01 00:40:17 -07:00
Daniel Han
682f16aa5a Update tokenizer_utils.py 2024-10-01 00:35:54 -07:00
Daniel Han
cd530425f0 Update tokenizer_utils.py 2024-10-01 00:20:01 -07:00
Daniel Han
3ac34b5071 Update tokenizer_utils.py 2024-10-01 00:14:52 -07:00
Daniel Han
e1b0b8f6ef Update chat_templates.py 2024-09-30 23:08:46 -07:00
Daniel Han
ccfb9b68ff Fix merges (#1079) 
* Layernorm

* Update layernorm.py

* Update layernorm.py

* Update layernorm.py

* Update layernorm.py

* Update layernorm.py

* Update layernorm.py

* Patch layernorm

* Update layernorm.py

* RMS Layernorm

* Update rms_layernorm.py

* Causal LM

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update layernorm.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update _utils.py

* Update _utils.py

* Llama 3.2

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update llama.py

* Update vision.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update loader.py

* Update loader.py

* Update loader.py

* Dependencies

* Update pyproject.toml

* Update _utils.py
 2024-09-30 03:03:01 -07:00
Daniel Han
cd7fe775fd Update _utils.py 2024-09-30 02:51:32 -07:00
Daniel Han
5147501121 Update pyproject.toml 2024-09-30 02:48:31 -07:00
Daniel Han
c8f45c8def Dependencies 2024-09-30 02:09:15 -07:00
Daniel Han
fa3b87e013 Update loader.py 2024-09-29 23:22:05 -07:00
Daniel Han
fe9171d15c Update loader.py 2024-09-29 23:15:22 -07:00
Daniel Han
98a1e16f57 Update loader.py 2024-09-29 23:13:44 -07:00
Daniel Han
679e2b601f Merge branch 'main' into nightly 2024-09-29 23:13:16 -07:00
Daniel Han
e694d3541f Update loader.py 2024-09-29 01:42:58 -07:00
Daniel Han
f28614023c Update pyproject.toml 2024-09-27 01:36:45 -07:00
Daniel Han
2171d2d4b1 Update tokenizer_utils.py 2024-09-26 01:23:40 -07:00
Daniel Han
6c78fc5a55 Update README.md 2024-09-26 00:12:42 -07:00
Daniel Han
b417295e95 Update README.md 2024-09-26 00:05:38 -07:00
Daniel Han
b6b640a67f Update README.md 2024-09-26 00:02:15 -07:00
Daniel Han
412963a078 Update pyproject.toml 2024-09-25 23:47:15 -07:00
Daniel Han
fb0ee5fc0c Update pyproject.toml 2024-09-25 23:13:49 -07:00
Daniel Han
703699134e Remove version checks 2024-09-25 23:00:09 -07:00
Daniel Han
83782024a0 Update _utils.py 2024-09-25 22:56:41 -07:00
Daniel Han
0472fba0a6 Update llama.py 2024-09-25 22:12:21 -07:00
Daniel Han
7ca67313d0 Update llama.py 2024-09-25 19:38:01 -07:00
Daniel Han
72de37e321 Update llama.py 2024-09-25 19:15:40 -07:00
Daniel Han
cbbff06973 Update llama.py 2024-09-25 18:24:30 -07:00
Daniel Han
dc85077f1f Update llama.py 2024-09-25 18:10:46 -07:00
Daniel Han
d4eed9807f Update llama.py 2024-09-25 18:07:21 -07:00