ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix/transformers-unpack (#1180)

Browse source 
* Fix DPO, ORPO (#1177)

* Fix TRL

* Update mistral.py

* Patch processing_class

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Installation guide (#1165)

* chore: update chat_templates.py (#1166)

orginal -> original

* Disable Flex Attention

* Update tokenizer_utils.py

* Update _utils.py

* n_items

* Update cross_entropy_loss.py

* Fix DPO, ORPO

* Update _utils.py

---------

Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>

* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
This commit is contained in:
Edd 2024-10-24 23:10:52 +04:00 • committed by GitHub
commit 79effae03d
1 changed files with 7 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

7
unsloth/kernels/cross_entropy_loss.py
View file

@ -388,6 +388,13 @@ from transformers.models.llama.modeling_llama import (
List,
Tuple,
)
try:
from transformers.models.llama.modeling_llama import Unpack, KwargsForCausalLM
except ImportError:
logger.warning("Unsloth: Could not find Unpack, KwargsForCausalLM in LlamaForCausalLM. "
"This is expected if you are using an older version of Transformers (<4.46.0). ")
import inspect, re
function = inspect.getsource(LlamaForCausalLM.forward)
function = function.split("\n")