Daniel Han
646f1b7fca
Update _utils.py
2024-11-04 17:56:16 -08:00
Daniel Han
766bf1ef65
Update _utils.py
2024-11-04 01:23:24 -08:00
Daniel Han
4b5847ffd5
Update _utils.py
2024-11-04 01:20:31 -08:00
Daniel Han
7d5111a40c
constexpr
2024-11-04 00:05:51 -08:00
Daniel Han
4b1d9e2622
constexpr
2024-11-04 00:03:24 -08:00
Daniel Han
9bae6e2149
Update _utils.py
2024-11-03 23:55:39 -08:00
Daniel Han
3b32d81f9e
int64
2024-11-03 23:53:03 -08:00
Daniel Han
0164087604
Update _utils.py
2024-11-03 18:13:11 -08:00
Daniel Han
608916a831
Update _utils.py
2024-11-03 17:56:03 -08:00
Daniel Han
013ebaa876
Post patch
2024-11-03 17:49:08 -08:00
Daniel Han
76da5117d3
Update _utils.py
2024-11-03 17:15:27 -08:00
Daniel Han
3d19a71cf9
Update _utils.py
2024-11-03 17:09:37 -08:00
Daniel Han
32eac0b6c5
Update _utils.py
2024-11-03 15:25:18 -08:00
Daniel Han
607ac34351
Update _utils.py
2024-11-03 15:21:03 -08:00
Daniel Han
1190ed45b3
CE Loss
2024-11-03 15:15:58 -08:00
Daniel Han
3f0e56fc23
Update _utils.py
2024-11-03 15:02:38 -08:00
Daniel Han
63a18286d7
Update _utils.py
2024-11-03 15:01:29 -08:00
Daniel Han
751413ed00
Patching
2024-11-03 01:18:48 -07:00
Daniel Han
35aa992615
Utils
2024-11-02 19:08:57 -07:00
Daniel Han
ad63a32a03
Tied weights
2024-10-31 16:25:26 -07:00
Daniel Han
526505c119
Update _utils.py
2024-10-30 14:01:49 -07:00
Daniel Han
74ab93c9da
Update _utils.py
2024-10-30 14:00:03 -07:00
Daniel Han
5b75e21a4b
Update _utils.py
2024-10-30 13:54:11 -07:00
Daniel Han
784dd13da7
Update _utils.py
2024-10-30 13:51:10 -07:00
Daniel Han
9ccbc0ed1f
Update _utils.py
2024-10-30 13:35:01 -07:00
Daniel Han
7e1692ace1
Bug fixes
2024-10-30 13:11:43 -07:00
Daniel Han
c6e9af2e5b
Update _utils.py
2024-10-28 10:41:31 -07:00
Daniel Han
5541ab48fe
Update _utils.py
2024-10-28 01:12:33 -07:00
Daniel Han
2dfdba3493
More patching
2024-10-28 01:10:23 -07:00
Daniel Han
02437a8391
Typo
2024-10-27 19:09:27 -07:00
Daniel Han
1c044da660
Fix pad token
2024-10-27 19:06:57 -07:00
Daniel Han
3acc5afad3
Update _utils.py
2024-10-27 17:34:33 -07:00
Daniel Han
7083a1d455
Unk token issues
2024-10-27 17:32:26 -07:00
Daniel Han
007efc2751
Update _utils.py
2024-10-27 15:09:35 -07:00
Daniel Han
d76eda4f66
Bug fixes ( #1195 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
* Update _utils.py
* fix/transformers-unpack (#1180 )
* Fix DPO, ORPO (#1177 )
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* donot upcast lm_head and embeddings to float32 (#1186 )
* Cleanup upcast logs (#1188 )
* Fix/phi-longrope (#1193 )
* Enhance rotary embedding handling in LlamaAttention and LongRopeRotaryEmbedding
* Typo
* Improve rotary embedding handling in LlamaAttention to prevent errors with short KV cache
* Update llama.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update transformers
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
2024-10-26 01:21:24 -07:00
Daniel Han
9ca13b836f
Update _utils.py
2024-10-24 12:17:48 -07:00
Daniel Han
9ca0bba14b
Fix 4.47 issue ( #1182 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
* Update _utils.py
* fix/transformers-unpack (#1180 )
* Fix DPO, ORPO (#1177 )
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
2024-10-24 12:17:21 -07:00
Daniel Han
4f1c474d4a
Fix DPO, ORPO ( #1177 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
2024-10-24 00:36:37 -07:00
Daniel Han
1e7e0e2368
Update _utils.py
2024-10-23 12:39:58 -07:00
Daniel Han
0e5a507f87
Many bug fixes ( #1162 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
2024-10-23 03:14:57 -07:00
Daniel Han
9327d90002
Update _utils.py
2024-10-18 23:10:30 -07:00
vo1d-ai
12bdd86636
fix: compute_loss bug ( #1151 )
...
Currently, Unsloth doesn't pass additional parameters to Trainer.compute_loss such as return_outputs. This leads to errors when calling trainer.evaluate(). This change fixes the bug by properly passing parameters to Trainer.compute_loss.
2024-10-18 20:46:07 -07:00
Daniel Han
3085f4c3da
Update _utils.py
2024-10-17 20:50:05 -07:00
Daniel Han
e210840ba9
Gradient Accumulation Fix ( #1146 )
...
* Unsloth Zoo
* Update trainer.py
* Update trainer.py
* Update cross_entropy_loss.py
* n_items
* Update llama.py
* kwargs
* Remove extraneous f prefixes (#1133 )
Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>
* Update __init__.py
* kwargs
* Update trainer.py
* Update trainer.py
* Update trainer.py
* Fix GA
* Update _utils.py
* Update llama.py
* Update tokenizer_utils.py
* Warn on old versions
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
---------
Co-authored-by: Emil Sadek <esadek@hotmail.com>
Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>
2024-10-17 20:43:07 -07:00
Daniel Han
38663b01f5
Gradient Accumulation Fix ( #1134 )
...
* Unsloth Zoo
* Update trainer.py
* Update trainer.py
* Update cross_entropy_loss.py
* n_items
* Update llama.py
* kwargs
* Remove extraneous f prefixes (#1133 )
Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>
* Update __init__.py
---------
Co-authored-by: Emil Sadek <esadek@hotmail.com>
Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>
2024-10-14 19:17:35 -07:00
Daniel Han
597e1894f6
Fix merges ( #1079 )
...
* Layernorm
* Update layernorm.py
* Update layernorm.py
* Update layernorm.py
* Update layernorm.py
* Update layernorm.py
* Update layernorm.py
* Patch layernorm
* Update layernorm.py
* RMS Layernorm
* Update rms_layernorm.py
* Causal LM
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update layernorm.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* Llama 3.2
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update llama.py
* Update vision.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update loader.py
* Update loader.py
* Update loader.py
* Dependencies
* Update pyproject.toml
* Update _utils.py
2024-09-30 03:03:01 -07:00
Daniel Han
f00eaf1b09
Update _utils.py
2024-09-25 22:56:41 -07:00
Daniel Han
087ffc1907
Llama 3.2 ( #1058 )
...
* Layernorm
* Update layernorm.py
* Update layernorm.py
* Update layernorm.py
* Update layernorm.py
* Update layernorm.py
* Update layernorm.py
* Patch layernorm
* Update layernorm.py
* RMS Layernorm
* Update rms_layernorm.py
* Causal LM
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update layernorm.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* Llama 3.2
2024-09-25 11:48:24 -07:00
Daniel Han
fb77505f84
Update _utils.py
2024-09-23 10:56:55 -07:00
Daniel Han
e5381b6a26
Update _utils.py
2024-09-22 02:38:28 -07:00
