Commit graph

85 commits

Author SHA1 Message Date
Daniel Han
bfe38e6ea8 Update README.md 2024-08-05 00:00:53 -07:00
Daniel Han
b43855fb36 Update README.md 2024-08-04 23:59:57 -07:00
Daniel Han
291bc6e254 Update README.md 2024-08-04 23:50:40 -07:00
emuchogu
46b4348698
pascal support (#870)
Co-authored-by: Edward Muchogu <muchogu@gmail.com>
2024-08-04 23:45:51 -07:00
Daniel Han
dfca5516e7 Update README.md 2024-07-31 09:50:11 -07:00
Daniel Han
b85670de83
Gemma (#843)
* bugs

* Update _utils.py

* flash-attn softcapping

* Update gemma2.py

* Update gemma2.py

* Update gemma2.py

* Update gemma2.py

* Update mapper.py

* Update README.md

* Update _utils.py
2024-07-31 08:54:58 -07:00
Daniel Han
a96d16e44b Update README.md 2024-07-23 15:08:09 -07:00
Daniel Han
824511e265 Update README.md 2024-07-23 12:07:27 -07:00
Daniel Han
22968a2134 Update README.md 2024-07-23 11:51:08 -07:00
Daniel Han
c86b13d465
Llama 3.1 (#797)
* Llama 3.1

* Update _utils.py

* Llama 3.1

* Update _utils.py

* Update llama.py

* Update llama.py

* hack for rotary

* patch RoPE

* refix rope

* Update _utils.py

* Update llama.py

* Llama 3.1 check

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py
2024-07-23 11:40:49 -07:00
Daniel Han
568bfddfc3 Update README.md 2024-07-19 03:05:15 -07:00
Daniel Han
39885a4ddc Update README.md 2024-07-19 03:03:50 -07:00
Daniel Han
5ab565fb2c
Gemma2 (#723)
* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

---------

Co-authored-by: Michael <107991372+shimmyshimmer@users.noreply.github.com>
2024-07-03 12:12:21 -07:00
Daniel Han
b34a44e969
Qwen bug fixes (#639)
* Update llama.py

* offload

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* continued pretraining trainer

* Update trainer.py

* Update trainer.py

* Update trainer.py

* Update trainer.py

* is_bfloat16_supported

* Update __init__.py

* Update README.md

* Update llama.py

* is_bfloat16_supported

* Update __init__.py

* Mistral v3

* Phi 3 medium

* Update chat_templates.py

* Update chat_templates.py

* Phi-3

* Update save.py

* Update README.md

Mistral v3 to Mistral v0.3

* Untrained tokens

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update llama.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update save.py

* Update save.py

* Update save.py

* checkpoint

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update llama.py

* accelerate

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* train_dataloader

* Update llama.py

* Update llama.py

* Update llama.py

* use_fast_convert

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* remove_special_tokens

* Ollama

* Update chat_templates.py

* Update chat_templates.py

* Update chat_templates.py

* Update llama.py

* Update chat_templates.py

* Support bfloat16 GGUF

* Update save.py

* Update llama.py

* fast_forward_inference

* Update mapper.py

* Update loader.py

* Update llama.py

* Update tokenizer_utils.py

* info

* edits

* Create chat template

* Fix tokenizer

* Update tokenizer_utils.py

* fix case where gguf saving fails due to first_conversion dtype (#630)

* Support revision parameter in FastLanguageModel.from_pretrained (#629)

* support `revision` parameter

* match unsloth formatting of named parameters

* clears any selected_adapters before calling internal_model.save_pretrained (#609)

* Update __init__.py (#602)

Check for incompatible modules before importing unsloth

* Fixed unsloth/tokenizer_utils.py for chat training (#604)

* Add GGML saving option to Unsloth for easier Ollama model creation and testing. (#345)

* Add save to llama.cpp GGML to save.py.

* Fix conversion command and path of convert to GGML function.

* Add autosaving lora to the GGML function

* Create lora save function for conversion to GGML

* Test fix #2 for saving lora

* Test fix #3 to save  the lora adapters to convert to GGML

* Remove unwated tokenizer saving for conversion to ggml and added a few print statements.

* Needed tokenizer for saving, added it back, also made it more unslothy style by having positional arguments, and added a few messages.

* Positional arguments didn't work out, so reverted to older version of the code, and added a few comments.

* Test fix 1 for arch

* Test fix 2 new Mistral error.

* Test fix 3

* Revert to old version for testing.

* Upload issue test fix 1

* Fix 2 uploading ggml

* Positional ags added.

* Temporray remove positional args

* Fix upload again!!!

* Add print statements and fix link

* Make the calling name better

* Create local saving for GGML

* Add choosing directory to save local GGML.

* Fix lil variable error in the save_to_custom_dir func

* docs: Add LoraConfig parameters documentation (#619)

* llama.cpp failing (#371)

llama.cpp is failing to generate quantize versions for the trained models.

Error:

```bash
You might have to compile llama.cpp yourself, then run this again.
You do not need to close this Python program. Run the following commands in a new terminal:
You must run this in the same folder as you're saving your model.
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make clean && LLAMA_CUDA=1 make all -j
Once that's done, redo the quantization.
```

But when i do clone this with recursive it works.

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* fix libcuda_dirs import for triton 3.0 (#227)

* fix libcuda_dirs import for triton 3.0

* Update __init__.py

* Update __init__.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update save.py

* Update __init__.py

* Update fast_lora.py

* Update save.py

* Update save.py

* Update save.py

* Update loader.py

* Update save.py

* Update save.py

* quantize now llama-quantize

* Update chat_templates.py

* Update loader.py

* Update mapper.py

* Update __init__.py

* embedding size

* Update qwen2.py

* docs

* Update README.md

* Update qwen2.py

* README: Fix minor typo. (#559)

* README: Fix minor typo.

One-character typo fix while reading.

* Update README.md

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update mistral.py

* Update qwen2.py

* Update qwen2.py

* Update qwen2.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update README.md

* FastMistralModel

---------

Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Eliot Hall <60240707+chrehall68@users.noreply.github.com>
Co-authored-by: Rickard Edén <rickardeden@gmail.com>
Co-authored-by: XiaoYang <xyangk@gmail.com>
Co-authored-by: Oseltamivir <58582368+Oseltamivir@users.noreply.github.com>
Co-authored-by: mahiatlinux <110882203+mahiatlinux@users.noreply.github.com>
Co-authored-by: Sébastien De Greef <sebdg@binarycompany.com>
Co-authored-by: Alberto Ferrer <albertof@barrahome.org>
Co-authored-by: Thomas Viehmann <tv.github-private@beamnet.de>
Co-authored-by: Walter Korman <lemurware@gmail.com>
2024-06-14 20:59:45 +10:00
Daniel Han-Chen
83807103ca Update README.md 2024-06-05 06:15:27 +10:00
Daniel Han-Chen
f9689b1472 Update README.md 2024-06-05 06:14:11 +10:00
Daniel Han
9515038867
Nightly (#548)
* Update llama.py

* offload

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* continued pretraining trainer

* Update trainer.py

* Update trainer.py

* Update trainer.py

* Update trainer.py

* is_bfloat16_supported

* Update __init__.py

* Update README.md

* Update llama.py

* is_bfloat16_supported

* Update __init__.py

* Mistral v3

* Phi 3 medium

* Update chat_templates.py

* Update chat_templates.py

* Phi-3

* Update save.py

* Update README.md

Mistral v3 to Mistral v0.3

* Untrained tokens

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update llama.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update save.py

* Update save.py

* Update save.py

* checkpoint

---------

Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
2024-05-29 00:30:38 +10:00
Daniel Han
2e1cb3888b
Phi-3, Llama-3 bug fixes (#519)
* Update llama.py

* offload

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* continued pretraining trainer

* Update trainer.py

* Update trainer.py

* Update trainer.py

* Update trainer.py

* is_bfloat16_supported

* Update __init__.py

* Update README.md

* Update llama.py

* is_bfloat16_supported

* Update __init__.py

* Mistral v3

* Phi 3 medium

* Update chat_templates.py

* Update chat_templates.py

* Phi-3
2024-05-24 06:36:10 +10:00
Daniel Han-Chen
a106e0b256 Update README.md 2024-05-23 04:31:26 +10:00
Daniel Han
5134a42f06
Nightly (#506)
* Update llama.py

* offload

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* continued pretraining trainer

* Update trainer.py

* Update trainer.py

* Update trainer.py

* Update trainer.py

* is_bfloat16_supported

* Update __init__.py

* Update README.md

* Update llama.py
2024-05-22 04:45:57 +10:00
Daniel Han
d4512f7c13
May 2024 Prelim (#447)
* Fix prompt

* Update chat_templates.py

* fix_untrained_tokens

* Update llama.py

* add tokens

* Update _utils.py

* Update tokenizer_utils.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* pad_token

* Update chat_templates.py

* Update chat_templates.py

* tokenizer

* Update save.py

* Update chat_templates.py

* Update chat_templates.py

* patch tokenizer padding

* Update tokenizer_utils.py

* Update save.py

* Fix: loading models with resized vocabulary (#377)

* new: vocab resize on load

* new: gitignore

* GGUF fix

* Readme (#390)

* Update README.md

* Update README.md

---------

Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>

* Update README.md

* Delete .gitignore

* Phi-3

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Fix reserved tokens

* Update save.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update chat_templates.py

* Update save.py

* Update _utils.py

* Update chat_templates.py

* Adds dependencies and extras for torch 2.3.0 with new xformers versions (#415)

* Adds dependencies and extras for torch 2.3.0 with new xformers versions

* Add 2.3.0 section to readme

* Support Qwen2 (#428)

* support Qwen2

* support Qwen2

* Delete README.md

* Revert "Delete README.md"

This reverts commit 026b05f859.

* Update README.md

* Qwen2 == Mistral

* Update llama.py

* Update __init__.py

* Update README.md

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update save.py

* Update save.py

* Update _utils.py

* Update save.py

* Update save.py

* Update save.py

* test_hf_gguf_equivalence

* Update chat_templates.py

* Update chat_templates.py

* --pad-vocab

* Update tokenizer_utils.py

---------

Co-authored-by: Igor Kilbas <whitemarsstudios@gmail.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Nathan Azrak <42650258+nathan-az@users.noreply.github.com>
Co-authored-by: Yang JianXin <995462226@qq.com>
2024-05-13 05:22:03 +10:00
Daniel Han-Chen
72aee4a3a1 Update README.md 2024-04-30 20:26:10 +10:00
Daniel Han
81819c337d
Phi-3 (#397)
* Fix prompt

* Update chat_templates.py

* fix_untrained_tokens

* Update llama.py

* add tokens

* Update _utils.py

* Update tokenizer_utils.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* pad_token

* Update chat_templates.py

* Update chat_templates.py

* tokenizer

* Update save.py

* Update chat_templates.py

* Update chat_templates.py

* patch tokenizer padding

* Update tokenizer_utils.py

* Update save.py

* Fix: loading models with resized vocabulary (#377)

* new: vocab resize on load

* new: gitignore

* GGUF fix

* Readme (#390)

* Update README.md

* Update README.md

---------

Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>

* Update README.md

* Delete .gitignore

* Phi-3

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

---------

Co-authored-by: Igor Kilbas <whitemarsstudios@gmail.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
2024-04-30 05:59:02 +10:00
Daniel Han
5039ea3c36
Nightly (#370)
* Fix prompt

* Update chat_templates.py

* fix_untrained_tokens

* Update llama.py

* add tokens

* Update _utils.py

* Update tokenizer_utils.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* pad_token

* Update chat_templates.py

* Update chat_templates.py

* tokenizer

* Update save.py

* Update chat_templates.py

* Update chat_templates.py

* patch tokenizer padding

* Update tokenizer_utils.py

* Update save.py

* Fix: loading models with resized vocabulary (#377)

* new: vocab resize on load

* new: gitignore

* GGUF fix

* Readme (#390)

* Update README.md

* Update README.md

---------

Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>

* Update README.md

* Delete .gitignore

---------

Co-authored-by: Igor Kilbas <whitemarsstudios@gmail.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
2024-04-29 04:47:03 +10:00
Daniel Han-Chen
ce523ed15c Update README.md 2024-04-20 14:22:52 +10:00
Daniel Han
88eee50267
Update README.md (#352) 2024-04-19 05:50:19 +10:00
Daniel Han
9c7bbecb50
Update README.md (#351) 2024-04-19 05:47:04 +10:00
Daniel Han
4606443b77
Readme Changes (#324)
* Update README.md

* Update README.md

---------

Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
2024-04-11 01:43:34 +10:00
Daniel Han
cb12179772
Bug fixes (#249)
* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update cross_entropy_loss.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* rope

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* llama

* Update llama.py

* gemma

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update save.py

* RoPE

* Update llama.py

* Update llama.py

* Update llama.py

* Update gemma.py

* correct_dtype

* Update gemma.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Chat Templates

* Update README.md

* Update README.md

* Update llama.py

* DoRA

* Update _utils.py

* Update chat_templates.py

* Update llama.py

* Hotfix - fix DoRA, Gemma prompt template (#202) (#203)

* Update save.py

* saving

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update __init__.py

* Update save.py

* Update save.py

* Update save.py

* save

* trainer

* spaces

* original

* Gemma

* Update pyproject.toml

* Update mapper.py

* Update fast_lora.py

* FastGemmaModel

* model_type

* Update llama.py

* Update llama.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update llama.py

* Update fast_lora.py

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* Update llama.py

* Update llama.py

* gemma

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update fast_lora.py

* Update fast_lora.py

* Fast CE Loss

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* CE

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* Update geglu.py

* Update cross_entropy_loss.py

* revert

* Update llama.py

* Update llama.py

* norm

* Update gemma.py

* Update gemma.py

* position_ids

* Update gemma.py

* Update gemma.py

* pos

* Update llama.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update cross_entropy_loss.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* revert

* revert

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update cross_entropy_loss.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* rope

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* llama

* Update llama.py

* gemma

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update save.py

* RoPE

* Update llama.py

* Update llama.py

* Update llama.py

* Update gemma.py

* correct_dtype

* Update gemma.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Chat Templates

* Update README.md

* Update README.md

* Update llama.py

* DoRA

* Update _utils.py

* Update chat_templates.py

* Update pyproject.toml

* Small fixes

* Update pyproject.toml

* Approx gelu

* Update geglu.py

* Approx gelu

* Update llama.py

* Update __init__.py

* Update __init__.py

* Update _utils.py

* Update geglu.py

* Update gemma.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Fix Gemma merging

* Update rms_layernorm.py

* Update gemma.py

* Update pyproject.toml

* Layernorms

* Gemma precision

* Update gemma.py

* sqrt

* Update gemma.py

* Update save.py

* RoPE and Gemma precision

* Update rms_layernorm.py

* Fix warning

* Update chat_templates.py

* Update chat_templates.py

* Update save.py

* Update save.py

* Update save.py

* Update chat_templates.py

* Update llama.py

* model_name

* Update loader.py

* Tokenizer overwritten

* Update llama.py

* Update llama.py

* Update llama.py

* Update save.py

* Accuracy

* Revert

* Update save.py

* Update fast_lora.py

* Update fast_lora.py

* Update fast_lora.py

* Update fast_lora.py

* Update fast_lora.py

* Update chat_templates.py

* Update save.py

* Update save.py

* Update llama.py

* Update llama.py

* Account for DoRA

* Update llama.py

* Update save.py

* GGUF incorrect

* Update save.py

* Update pyproject.toml

* kaggle new

* Update pyproject.toml

* Update pyproject.toml

* upcasting

* Fix Colab

* Update pyproject.toml

* Update pyproject.toml

* Update pyproject.toml

* Update pyproject.toml

* Update pyproject.toml

* Update pyproject.toml

* Update pyproject.toml

* Update pyproject.toml

* Update chat_templates.py

* Update chat_templates.py

* Update chat_templates.py

* Update chat_templates.py

* Update chat_templates.py

* Update pyproject.toml

* Update pyproject.toml

* Update pyproject.toml

* Update rope_embedding.py

* Update rope_embedding.py

* Fix bugs

* Update fast_lora.py

* Update fast_lora.py

* Update README.md

* Update README.md

* GGUF

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update README.md

* Update README.md
2024-03-17 02:47:05 +11:00
Daniel Han
42076f6580
Fix Gemma GGUF (#234)
* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* revert

* revert

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update cross_entropy_loss.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* rope

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* llama

* Update llama.py

* gemma

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update save.py

* RoPE

* Update llama.py

* Update llama.py

* Update llama.py

* Update gemma.py

* correct_dtype

* Update gemma.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Chat Templates

* Update README.md

* Update README.md

* Update llama.py

* DoRA

* Update _utils.py

* Update chat_templates.py

* Update llama.py

* Hotfix - fix DoRA, Gemma prompt template (#202) (#203)

* Update save.py

* saving

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update __init__.py

* Update save.py

* Update save.py

* Update save.py

* save

* trainer

* spaces

* original

* Gemma

* Update pyproject.toml

* Update mapper.py

* Update fast_lora.py

* FastGemmaModel

* model_type

* Update llama.py

* Update llama.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update llama.py

* Update fast_lora.py

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* Update llama.py

* Update llama.py

* gemma

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update fast_lora.py

* Update fast_lora.py

* Fast CE Loss

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* CE

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* Update geglu.py

* Update cross_entropy_loss.py

* revert

* Update llama.py

* Update llama.py

* norm

* Update gemma.py

* Update gemma.py

* position_ids

* Update gemma.py

* Update gemma.py

* pos

* Update llama.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update cross_entropy_loss.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* revert

* revert

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update cross_entropy_loss.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* rope

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* llama

* Update llama.py

* gemma

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update save.py

* RoPE

* Update llama.py

* Update llama.py

* Update llama.py

* Update gemma.py

* correct_dtype

* Update gemma.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Chat Templates

* Update README.md

* Update README.md

* Update llama.py

* DoRA

* Update _utils.py

* Update chat_templates.py

* Update pyproject.toml

* Small fixes

* Update pyproject.toml

* Approx gelu

* Update geglu.py

* Approx gelu

* Update llama.py

* Update __init__.py

* Update __init__.py

* Update _utils.py

* Update geglu.py

* Update gemma.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Fix Gemma merging

* Update rms_layernorm.py

* Update gemma.py

* Update pyproject.toml

* Layernorms

* Gemma precision

* Update gemma.py

* sqrt

* Update gemma.py

* Update save.py

* RoPE and Gemma precision

* Update rms_layernorm.py

* Fix warning

* Update chat_templates.py

* Update chat_templates.py

* Update save.py

* Update save.py

* Update save.py

* Update chat_templates.py

* Update llama.py

* model_name

* Update loader.py

* Tokenizer overwritten

* Update llama.py

* Update llama.py

* Update llama.py

* Update save.py

* Accuracy

* Revert

* Update save.py

* Update fast_lora.py

* Update fast_lora.py

* Update fast_lora.py

* Update fast_lora.py

* Update fast_lora.py

* Update chat_templates.py

* Update save.py

* Update save.py

* Update llama.py

* Update llama.py

* Account for DoRA

* Update llama.py

* Update save.py

* GGUF incorrect

* Update save.py

* Update pyproject.toml

* kaggle new

* Update pyproject.toml

* Update pyproject.toml

* upcasting

* Fix Colab

* Update pyproject.toml

* Update pyproject.toml

* Update pyproject.toml

* Update pyproject.toml

* Update pyproject.toml

* Update pyproject.toml

* Update pyproject.toml

* Update pyproject.toml
2024-03-14 20:32:04 +11:00
Daniel Han
f946bed7b3
2.4x faster Gemma (#197)
* Update save.py

* Update save.py

* linking

* llama.cpp bugs

* Update save.py

* Update save.py

* saving

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update __init__.py

* Update save.py

* Update save.py

* Update save.py

* save

* trainer

* spaces

* original

* Gemma

* Update pyproject.toml

* Update mapper.py

* Update fast_lora.py

* FastGemmaModel

* model_type

* Update llama.py

* Update llama.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update llama.py

* Update fast_lora.py

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* Update llama.py

* Update llama.py

* gemma

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update fast_lora.py

* Update fast_lora.py

* Fast CE Loss

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* CE

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* Update geglu.py

* Update cross_entropy_loss.py

* revert

* Update llama.py

* Update llama.py

* norm

* Update gemma.py

* Update gemma.py

* position_ids

* Update gemma.py

* Update gemma.py

* pos

* Update llama.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update cross_entropy_loss.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* revert

* revert

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update cross_entropy_loss.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* rope

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* llama

* Update llama.py

* gemma

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update save.py

* RoPE

* Update llama.py

* Update llama.py

* Update llama.py

* Update gemma.py

* correct_dtype

* Update gemma.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Chat Templates

* Update README.md

* Update README.md
2024-02-27 01:42:10 +11:00
Daniel Han
3e4c5a323c
Feb 2024 Release (#187)
* Fast inference repatch

* Update llama.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update mistral.py

* Update __init__.py

* Fix inference

* Update mistral.py

* fast lm_head

* Remove fast path

* Update rope_embedding.py

* Update loader.py

* LlamaAttention_fast_forward_inference

* if past_key_value is not None and q_len == 1:

* revert inference

* Update loader.py

* past_key_value

* Update llama.py

* Update llama.py

* Fix SDPA

* Update llama.py

* padding

* Inference

* Update llama.py

* Revert

* Update mistral.py

* faster inference

* inference

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* inference

* Update llama.py

* Update utils.py

* faster inference

* Update llama.py

* revert

* lm_head

* Update llama.py

* inference

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update mistral.py

* Update llama.py

* faster inference

* Update llama.py

* fast inference

* Update llama.py

* Update llama.py

* Update mistral.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* torch compile

* past_key_values

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update llama.py

* fast inference + saving config.json

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update mistral.py

* fast inference again

* more temp matrices

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* fast inference

* Update mistral.py

* Update llama.py

* SDPA

* attention_mask

* New version

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update utils.py

* Update utils.py

* Update save.py

* Update save.py

* Torch 2.2.0

* Update save.py

* mistral swa

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Fix SWA inference

* Fix llm_int8_skip_modules

* SWA inference

* Update save.py

* Update save.py

* Update pyproject.toml

* __version__

* __version__

* Update save.py

* Update save.py

* Update mistral.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Chat Templates

* Update chat_templates.py

* Update chat_templates.py

* Update chat_templates.py

* Update chat_templates.py

* patch tokenizer

* Update chat_templates.py

* Saving, LlamaRotaryEmbedding issues

* Update llama.py

* Update mistral.py

* Update mapper.py

* Fix RoPE precision issues

* Bugs

* saving bugs

* Update llama.py

* readme

* spaces

* spaces

* globals

* slash

* slashes

* spaces

* apache

* Update save.py

* Update save.py

* Update loader.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* trainer

* Update save.py

* Update pyproject.toml

* install

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* PeftModel token + saving

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* linking

* llama.cpp bugs

* Update save.py

* Update save.py

* saving

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update __init__.py

* Update save.py

* Update save.py

* Update save.py

* save

* trainer

* spaces

* original
2024-02-21 03:58:59 +11:00
Daniel Han
a030e80203
Prelim Feb release (#173)
* Works?

* Update pyproject.toml

* Update fast_lora.py

* Update fast_lora.py

* Update fast_lora.py

* Update fast_lora.py

* Update fast_lora.py

* Swiglu

* Update swiglu.py

* Update fast_lora.py

* Update fast_lora.py

* Update fast_lora.py

* Update fast_lora.py

* Update swiglu.py

* Update fast_lora.py

* Update fast_lora.py

* Update fast_lora.py

* Update fast_lora.py

* Update fast_lora.py

* Update fast_lora.py

* Update fast_lora.py

* Update fast_lora.py

* attention_mask

* Update llama.py

* Update llama.py

* labels

* Update mistral.py

* Update llama.py

* attention mask

* Update save.py

* Update save.py

* Update mistral.py

* attention mask

* Update llama.py

* Update llama.py

* Update mistral.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update dpo.py

* Patch saving

* Update save.py

* Update save.py

* patch_saving_functions

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* print

* Mistral patch

* Update mistral.py

* Update save.py

* saving

* Update llama.py

* Update llama.py

* Fast inference repatch

* Update llama.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update mistral.py

* Update __init__.py

* Fix inference

* Update mistral.py

* fast lm_head

* Remove fast path

* Update rope_embedding.py

* Update loader.py

* LlamaAttention_fast_forward_inference

* if past_key_value is not None and q_len == 1:

* revert inference

* Update loader.py

* past_key_value

* Update llama.py

* Update llama.py

* Fix SDPA

* Update llama.py

* padding

* Inference

* Update llama.py

* Revert

* Update mistral.py

* faster inference

* inference

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* inference

* Update llama.py

* Update utils.py

* faster inference

* Update llama.py

* revert

* lm_head

* Update llama.py

* inference

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update mistral.py

* Update llama.py

* faster inference

* Update llama.py

* fast inference

* Update llama.py

* Update llama.py

* Update mistral.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* torch compile

* past_key_values

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update llama.py

* fast inference + saving config.json

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update mistral.py

* fast inference again

* more temp matrices

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* fast inference

* Update mistral.py

* Update llama.py

* SDPA

* attention_mask

* New version

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update utils.py

* Update utils.py

* Update save.py

* Update save.py

* Torch 2.2.0

* Update save.py

* mistral swa

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Fix SWA inference

* Fix llm_int8_skip_modules

* SWA inference

* Update save.py

* Update save.py

* Update pyproject.toml

* __version__

* __version__

* Update save.py

* Update save.py

* Update mistral.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Chat Templates

* Update chat_templates.py

* Update chat_templates.py

* Update chat_templates.py

* Update chat_templates.py

* patch tokenizer

* Update chat_templates.py

* Saving, LlamaRotaryEmbedding issues

* Update llama.py

* Update mistral.py
2024-02-15 00:07:42 +11:00
Daniel Han
e091bca34a
Update README.md (#165) 2024-02-09 15:59:17 +11:00
Daniel Han
4a95362cc3
Update README.md (#164) 2024-02-09 15:49:09 +11:00
Daniel Han
7e7f5f355f
Update README.md (#162) 2024-02-08 13:11:54 +11:00
Michael Han
1d393f7406
ReadMe Revamp (#156)
* HF Perf Button

* Update README.md

Adding new buttons cleanup

* Update README.md

* Delete images/Discord.png

* Delete images/try live demo green.png

* new transparent logos

* Revamping page

* Revamp mainpage

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* finetune button

* Delete start free finetune button.png

* free finetune button

* Add files via upload

* Update README.md

* Update README.md

* Add files via upload

* Add files via upload

* Update README.md

* Add files via upload

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Squashed commit of the following:

commit 35f2ab4a8b
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Sun Feb 4 17:35:56 2024 +1100

    2x faster inference (#151)

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update save.py

    * Update fast_lora.py

    * Update utils.py

    * Update llama.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update save.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Revert "Update llama.py"

    This reverts commit a208ec46e0.

    * Update llama.py

    * Works?

    * Update pyproject.toml

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Swiglu

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * attention_mask

    * Update llama.py

    * Update llama.py

    * labels

    * Update mistral.py

    * Update llama.py

    * attention mask

    * Update save.py

    * Update save.py

    * Update mistral.py

    * attention mask

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update dpo.py

    * Patch saving

    * Update save.py

    * Update save.py

    * patch_saving_functions

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * print

    * Mistral patch

    * Update mistral.py

    * Update save.py

    * saving

    * Update llama.py

    * Update llama.py

    * Fast inference repatch

    * Update llama.py

    * Update utils.py

    * Update utils.py

    * Update utils.py

    * Update mistral.py

    * Update __init__.py

    * Fix inference

    * Update mistral.py

    * fast lm_head

    * Remove fast path

    * Update rope_embedding.py

    * Update loader.py

    * LlamaAttention_fast_forward_inference

    * if past_key_value is not None and q_len == 1:

    * revert inference

    * Update loader.py

    * past_key_value

    * Update llama.py

    * Update llama.py

    * Fix SDPA

    * Update llama.py

    * padding

    * Inference

    * Update llama.py

    * Revert

    * Update mistral.py

    * faster inference

    * inference

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * inference

    * Update llama.py

    * Update utils.py

    * faster inference

    * Update llama.py

    * revert

    * lm_head

    * Update llama.py

    * inference

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * faster inference

    * Update llama.py

    * fast inference

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * torch compile

    * past_key_values

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update utils.py

    * Update utils.py

    * Update utils.py

    * Update utils.py

    * Update llama.py

    * fast inference + saving config.json

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * fast inference again

    * more temp matrices

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * fast inference

    * Update mistral.py

    * Update llama.py

    * SDPA

    * attention_mask

    * New version

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update utils.py

    * Update utils.py

commit 051a73b0e6
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Wed Jan 31 04:03:37 2024 +1100

    Hotfix - fix inference (#146)

    * faster saving & inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * fast inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Mistral correct RoPE scaling

    * Max sequence lengths

    * Apache 2

    * fast_linear_forward

    * Update utils.py

    * Update utils.py

    * No print

    * Update utils.py

    * Update utils.py

    * inference

    * Update llama.py

    * Fast inference RoPE

    * Update llama.py

    * Update llama.py

    * RoPE

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * LoRA

    * Fast LoRA saving

    * Update llama.py

    * hidden_states

    * q_len == 1

    * q_len issue

    * Update mistral.py

    * Update mistral.py

    * incorrect inference

    * Update to transformers 4.37

    * Graceful FA2 error + torch 2.1.1

    * Update mapper.py

    * Update pyproject.toml

    * Fix saving and bnb-4bit

    * Update fast_lora.py

    * Update fast_lora.py

    * remove patching

    * Update llama.py

    * Update llama.py

    * Update swiglu.py

    * Repatch

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update save.py

    * Update fast_lora.py

    * Update utils.py

    * Update llama.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update save.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Revert "Update llama.py"

    This reverts commit a208ec46e0.

    * Update llama.py

    * Works?

    * Update pyproject.toml

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Swiglu

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * attention_mask

    * Update llama.py

    * Update llama.py

    * labels

    * Update mistral.py

    * Update llama.py

    * attention mask

    * Update save.py

    * Update save.py

    * Update mistral.py

    * attention mask

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update dpo.py

    * Patch saving

    * Update save.py

    * Update save.py

    * patch_saving_functions

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * print

    * Mistral patch

    * Update mistral.py

    * Update save.py

    * saving

    * Update llama.py

    * Update llama.py

    * Fast inference repatch

    * Update llama.py

    * Update utils.py

    * Update utils.py

    * Update utils.py

    * Update mistral.py

    * Update __init__.py

    * Fix inference

    * Update mistral.py

    * fast lm_head

    * Remove fast path

    * Update rope_embedding.py

    * Update loader.py

    * LlamaAttention_fast_forward_inference

    * if past_key_value is not None and q_len == 1:

    * revert inference

    * Update loader.py

    * past_key_value

commit 0562464280
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Mon Jan 29 17:49:54 2024 +1100

    Fix inference attention mask (#142)

    * faster saving & inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * fast inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Mistral correct RoPE scaling

    * Max sequence lengths

    * Apache 2

    * fast_linear_forward

    * Update utils.py

    * Update utils.py

    * No print

    * Update utils.py

    * Update utils.py

    * inference

    * Update llama.py

    * Fast inference RoPE

    * Update llama.py

    * Update llama.py

    * RoPE

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * LoRA

    * Fast LoRA saving

    * Update llama.py

    * hidden_states

    * q_len == 1

    * q_len issue

    * Update mistral.py

    * Update mistral.py

    * incorrect inference

    * Update to transformers 4.37

    * Graceful FA2 error + torch 2.1.1

    * Update mapper.py

    * Update pyproject.toml

    * Fix saving and bnb-4bit

    * Update fast_lora.py

    * Update fast_lora.py

    * remove patching

    * Update llama.py

    * Update llama.py

    * Update swiglu.py

    * Repatch

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update save.py

    * Update fast_lora.py

    * Update utils.py

    * Update llama.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update save.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Revert "Update llama.py"

    This reverts commit a208ec46e0.

    * Update llama.py

    * Works?

    * Update pyproject.toml

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Swiglu

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * attention_mask

    * Update llama.py

    * Update llama.py

    * labels

    * Update mistral.py

    * Update llama.py

    * attention mask

    * Update save.py

    * Update save.py

    * Update mistral.py

    * attention mask

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update dpo.py

    * Patch saving

    * Update save.py

    * Update save.py

    * patch_saving_functions

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * print

    * Mistral patch

    * Update mistral.py

    * Update save.py

    * saving

    * Update llama.py

    * Update llama.py

commit 206a9b65f0
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Mon Jan 29 03:45:07 2024 +1100

    Nightly (#140)

    * faster saving & inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * fast inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Mistral correct RoPE scaling

    * Max sequence lengths

    * Apache 2

    * fast_linear_forward

    * Update utils.py

    * Update utils.py

    * No print

    * Update utils.py

    * Update utils.py

    * inference

    * Update llama.py

    * Fast inference RoPE

    * Update llama.py

    * Update llama.py

    * RoPE

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * LoRA

    * Fast LoRA saving

    * Update llama.py

    * hidden_states

    * q_len == 1

    * q_len issue

    * Update mistral.py

    * Update mistral.py

    * incorrect inference

    * Update to transformers 4.37

    * Graceful FA2 error + torch 2.1.1

    * Update mapper.py

    * Update pyproject.toml

    * Fix saving and bnb-4bit

    * Update fast_lora.py

    * Update fast_lora.py

    * remove patching

    * Update llama.py

    * Update llama.py

    * Update swiglu.py

    * Repatch

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update save.py

    * Update fast_lora.py

    * Update utils.py

    * Update llama.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update save.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Revert "Update llama.py"

    This reverts commit a208ec46e0.

    * Update llama.py

    * Works?

    * Update pyproject.toml

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Swiglu

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * attention_mask

    * Update llama.py

    * Update llama.py

    * labels

    * Update mistral.py

    * Update llama.py

    * attention mask

    * Update save.py

    * Update save.py

    * Update mistral.py

    * attention mask

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update dpo.py

    * Patch saving

    * Update save.py

    * Update save.py

    * patch_saving_functions

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * print

    * Mistral patch

    * Update mistral.py

    * Update save.py

    * saving

commit 8faf469f02
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Mon Jan 29 02:52:39 2024 +1100

    Fix saving issues (#139)

    * faster saving & inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * fast inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Mistral correct RoPE scaling

    * Max sequence lengths

    * Apache 2

    * fast_linear_forward

    * Update utils.py

    * Update utils.py

    * No print

    * Update utils.py

    * Update utils.py

    * inference

    * Update llama.py

    * Fast inference RoPE

    * Update llama.py

    * Update llama.py

    * RoPE

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * LoRA

    * Fast LoRA saving

    * Update llama.py

    * hidden_states

    * q_len == 1

    * q_len issue

    * Update mistral.py

    * Update mistral.py

    * incorrect inference

    * Update to transformers 4.37

    * Graceful FA2 error + torch 2.1.1

    * Update mapper.py

    * Update pyproject.toml

    * Fix saving and bnb-4bit

    * Update fast_lora.py

    * Update fast_lora.py

    * remove patching

    * Update llama.py

    * Update llama.py

    * Update swiglu.py

    * Repatch

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update save.py

    * Update fast_lora.py

    * Update utils.py

    * Update llama.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update save.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Revert "Update llama.py"

    This reverts commit a208ec46e0.

    * Update llama.py

    * Works?

    * Update pyproject.toml

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Swiglu

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * attention_mask

    * Update llama.py

    * Update llama.py

    * labels

    * Update mistral.py

    * Update llama.py

    * attention mask

    * Update save.py

    * Update save.py

    * Update mistral.py

    * attention mask

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update dpo.py

    * Patch saving

    * Update save.py

    * Update save.py

    * patch_saving_functions

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * print

commit 1ecc0185a5
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Sun Jan 28 04:30:29 2024 +1100

    1 more bug (#138)

    * faster saving & inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * fast inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Mistral correct RoPE scaling

    * Max sequence lengths

    * Apache 2

    * fast_linear_forward

    * Update utils.py

    * Update utils.py

    * No print

    * Update utils.py

    * Update utils.py

    * inference

    * Update llama.py

    * Fast inference RoPE

    * Update llama.py

    * Update llama.py

    * RoPE

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * LoRA

    * Fast LoRA saving

    * Update llama.py

    * hidden_states

    * q_len == 1

    * q_len issue

    * Update mistral.py

    * Update mistral.py

    * incorrect inference

    * Update to transformers 4.37

    * Graceful FA2 error + torch 2.1.1

    * Update mapper.py

    * Update pyproject.toml

    * Fix saving and bnb-4bit

    * Update fast_lora.py

    * Update fast_lora.py

    * remove patching

    * Update llama.py

    * Update llama.py

    * Update swiglu.py

    * Repatch

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update save.py

    * Update fast_lora.py

    * Update utils.py

    * Update llama.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update save.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Revert "Update llama.py"

    This reverts commit a208ec46e0.

    * Update llama.py

    * Works?

    * Update pyproject.toml

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Swiglu

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * attention_mask

    * Update llama.py

    * Update llama.py

    * labels

    * Update mistral.py

    * Update llama.py

    * attention mask

    * Update save.py

    * Update save.py

commit cd32ba76b7
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Sun Jan 28 04:20:06 2024 +1100

    Fix bugs + more accurate Swiglu (#137)

    * faster saving & inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * fast inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Mistral correct RoPE scaling

    * Max sequence lengths

    * Apache 2

    * fast_linear_forward

    * Update utils.py

    * Update utils.py

    * No print

    * Update utils.py

    * Update utils.py

    * inference

    * Update llama.py

    * Fast inference RoPE

    * Update llama.py

    * Update llama.py

    * RoPE

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * LoRA

    * Fast LoRA saving

    * Update llama.py

    * hidden_states

    * q_len == 1

    * q_len issue

    * Update mistral.py

    * Update mistral.py

    * incorrect inference

    * Update to transformers 4.37

    * Graceful FA2 error + torch 2.1.1

    * Update mapper.py

    * Update pyproject.toml

    * Fix saving and bnb-4bit

    * Update fast_lora.py

    * Update fast_lora.py

    * remove patching

    * Update llama.py

    * Update llama.py

    * Update swiglu.py

    * Repatch

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update save.py

    * Update fast_lora.py

    * Update utils.py

    * Update llama.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update save.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Revert "Update llama.py"

    This reverts commit a208ec46e0.

    * Update llama.py

    * Works?

    * Update pyproject.toml

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Swiglu

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * attention_mask

    * Update llama.py

    * Update llama.py

    * labels

    * Update mistral.py

    * Update llama.py

    * attention mask

commit 89daa0efcc
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Sat Jan 27 04:50:22 2024 +1100

    Inference bug fix (#134)

    * faster saving & inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * fast inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Mistral correct RoPE scaling

    * Max sequence lengths

    * Apache 2

    * fast_linear_forward

    * Update utils.py

    * Update utils.py

    * No print

    * Update utils.py

    * Update utils.py

    * inference

    * Update llama.py

    * Fast inference RoPE

    * Update llama.py

    * Update llama.py

    * RoPE

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * LoRA

    * Fast LoRA saving

    * Update llama.py

    * hidden_states

    * q_len == 1

    * q_len issue

    * Update mistral.py

    * Update mistral.py

    * incorrect inference

    * Update to transformers 4.37

    * Graceful FA2 error + torch 2.1.1

    * Update mapper.py

    * Update pyproject.toml

    * Fix saving and bnb-4bit

    * Update fast_lora.py

    * Update fast_lora.py

    * remove patching

    * Update llama.py

    * Update llama.py

    * Update swiglu.py

    * Repatch

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update save.py

    * Update fast_lora.py

    * Update utils.py

    * Update llama.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update save.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Revert "Update llama.py"

    This reverts commit a208ec46e0.

    * Update llama.py

commit 87a7ef1049
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Sat Jan 27 04:47:54 2024 +1100

    More bug fixes (#133)

    * faster saving & inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * fast inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Mistral correct RoPE scaling

    * Max sequence lengths

    * Apache 2

    * fast_linear_forward

    * Update utils.py

    * Update utils.py

    * No print

    * Update utils.py

    * Update utils.py

    * inference

    * Update llama.py

    * Fast inference RoPE

    * Update llama.py

    * Update llama.py

    * RoPE

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * LoRA

    * Fast LoRA saving

    * Update llama.py

    * hidden_states

    * q_len == 1

    * q_len issue

    * Update mistral.py

    * Update mistral.py

    * incorrect inference

    * Update to transformers 4.37

    * Graceful FA2 error + torch 2.1.1

    * Update mapper.py

    * Update pyproject.toml

    * Fix saving and bnb-4bit

    * Update fast_lora.py

    * Update fast_lora.py

    * remove patching

    * Update llama.py

    * Update llama.py

    * Update swiglu.py

    * Repatch

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update llama.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Update save.py

    * Update fast_lora.py

    * Update utils.py

    * Update llama.py

    * Update fast_lora.py

    * Update swiglu.py

    * Update save.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

commit 3d67790901
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Fri Jan 26 04:19:17 2024 +1100

    Fix bugs (#129)

    * faster saving & inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * fast inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Mistral correct RoPE scaling

    * Max sequence lengths

    * Apache 2

    * fast_linear_forward

    * Update utils.py

    * Update utils.py

    * No print

    * Update utils.py

    * Update utils.py

    * inference

    * Update llama.py

    * Fast inference RoPE

    * Update llama.py

    * Update llama.py

    * RoPE

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * LoRA

    * Fast LoRA saving

    * Update llama.py

    * hidden_states

    * q_len == 1

    * q_len issue

    * Update mistral.py

    * Update mistral.py

    * incorrect inference

    * Update to transformers 4.37

    * Graceful FA2 error + torch 2.1.1

    * Update mapper.py

    * Update pyproject.toml

    * Fix saving and bnb-4bit

    * Update fast_lora.py

    * Update fast_lora.py

    * remove patching

    * Update llama.py

    * Update llama.py

    * Update swiglu.py

    * Repatch

    * Update fast_lora.py

commit a833f40346
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Tue Jan 23 03:55:24 2024 +1100

    2-4x faster native HF inference (#119)

    * faster saving & inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * fast inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Mistral correct RoPE scaling

    * Max sequence lengths

    * Apache 2

    * fast_linear_forward

    * Update utils.py

    * Update utils.py

    * No print

    * Update utils.py

    * Update utils.py

    * inference

    * Update llama.py

    * Fast inference RoPE

    * Update llama.py

    * Update llama.py

    * RoPE

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * LoRA

    * Fast LoRA saving

commit b370c9c8aa
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Sun Jan 21 22:20:22 2024 +1100

    Hotfix (#118)

    * faster saving & inference

    * Update llama.py

    * Update save.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update mistral.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

commit 57a5b5a49d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 21 05:00:37 2024 +1100

    Update save.py

commit 5145a61e69
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 21 04:21:54 2024 +1100

    Update save.py

commit a7bd8d119c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 21 04:13:03 2024 +1100

    Update save.py

commit be4b97e7d8
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Sun Jan 21 03:43:49 2024 +1100

    Fixed saving! (#113)

    * Fix tokenizer, dropout, bias for LoRA

    * Update loader.py

    * Fix LoRA downcasting

    * Update _utils.py

    * Saving to GGUF

    * fix

    * colab_quantize_to_gguf

    * move save modules

    * save module

    * Update __init__.py

    * Update save.py

    * Temp downgrade due to TRL issue

    * Fix up bugs

    * Faster saving + other changes

    * Update llama.py

    * Saving modules

    * spelling

    * Update llama.py

    * Update save.py

    * Update save.py

    * Update loader.py

    * Update llama.py

    * patch saving

    * Update save.py

    * Update save.py

    * Update save.py

    * patch saving

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * original_model

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * saving to RAM leakage?

    * Update save.py

    * new_save_directory

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update pyproject.toml

    * Update pyproject.toml

    * Update pyproject.toml

    * Quick fixes

    * Update llama.py

    * Update llama.py

    * Update dpo.py

    * Update dpo.py

    * Update llama.py

    * Update save.py

    * getattr

    * RSLoRA and LoftQ direct support

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Fix DPO + GGUF

    * Fix quantization_method

    * Fix quantization_config

    * patch model

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update save.py

    * Update save.py

    * tokenizer_save_settings

    * Update save.py

    * quantization and loftq

    * Update save.py

    * Update llama.py

    * Update save.py

    * upload_to_huggingface

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

commit abb462be71
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Sat Jan 20 23:23:00 2024 +1100

    Hotfix for Jan 2024 Release (#110)

    * Fix tokenizer, dropout, bias for LoRA

    * Update loader.py

    * Fix LoRA downcasting

    * Update _utils.py

    * Saving to GGUF

    * fix

    * colab_quantize_to_gguf

    * move save modules

    * save module

    * Update __init__.py

    * Update save.py

    * Temp downgrade due to TRL issue

    * Fix up bugs

    * Faster saving + other changes

    * Update llama.py

    * Saving modules

    * spelling

    * Update llama.py

    * Update save.py

    * Update save.py

    * Update loader.py

    * Update llama.py

    * patch saving

    * Update save.py

    * Update save.py

    * Update save.py

    * patch saving

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * original_model

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * saving to RAM leakage?

    * Update save.py

    * new_save_directory

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update pyproject.toml

    * Update pyproject.toml

    * Update pyproject.toml

    * Quick fixes

    * Update llama.py

    * Update llama.py

    * Update dpo.py

    * Update dpo.py

    * Update llama.py

    * Update save.py

    * getattr

    * RSLoRA and LoftQ direct support

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Fix DPO + GGUF

    * Fix quantization_method

    * Fix quantization_config

    * patch model

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update save.py

    * Update save.py

    * tokenizer_save_settings

    * Update save.py

    * quantization and loftq

    * Update save.py

    * Update llama.py

    * Update save.py

commit 31e2d71720
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Sat Jan 20 04:25:06 2024 +1100

    Quick fixes (#106)

    * Fix tokenizer, dropout, bias for LoRA

    * Update loader.py

    * Fix LoRA downcasting

    * Update _utils.py

    * Saving to GGUF

    * fix

    * colab_quantize_to_gguf

    * move save modules

    * save module

    * Update __init__.py

    * Update save.py

    * Temp downgrade due to TRL issue

    * Fix up bugs

    * Faster saving + other changes

    * Update llama.py

    * Saving modules

    * spelling

    * Update llama.py

    * Update save.py

    * Update save.py

    * Update loader.py

    * Update llama.py

    * patch saving

    * Update save.py

    * Update save.py

    * Update save.py

    * patch saving

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * original_model

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * saving to RAM leakage?

    * Update save.py

    * new_save_directory

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update pyproject.toml

    * Update pyproject.toml

    * Update pyproject.toml

    * Quick fixes

    * Update llama.py

    * Update llama.py

    * Update dpo.py

    * Update dpo.py

    * Update llama.py

    * Update save.py

    * getattr

    * RSLoRA and LoftQ direct support

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Fix DPO + GGUF

commit 8846337e5c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 20 02:30:31 2024 +1100

    Update _utils.py

commit d378df87e5
Merge: c1e7480 920e3c2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 19 23:15:38 2024 +1100

    Merge branch 'main' of https://github.com/unslothai/unsloth

commit c1e7480ac2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 19 23:15:20 2024 +1100

    Revert quantization methods

commit 920e3c2ea0
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Fri Jan 19 22:57:22 2024 +1100

    getattr issues (#103)

    * Fix tokenizer, dropout, bias for LoRA

    * Update loader.py

    * Fix LoRA downcasting

    * Update _utils.py

    * Saving to GGUF

    * fix

    * colab_quantize_to_gguf

    * move save modules

    * save module

    * Update __init__.py

    * Update save.py

    * Temp downgrade due to TRL issue

    * Fix up bugs

    * Faster saving + other changes

    * Update llama.py

    * Saving modules

    * spelling

    * Update llama.py

    * Update save.py

    * Update save.py

    * Update loader.py

    * Update llama.py

    * patch saving

    * Update save.py

    * Update save.py

    * Update save.py

    * patch saving

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * original_model

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * saving to RAM leakage?

    * Update save.py

    * new_save_directory

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update pyproject.toml

    * Update pyproject.toml

    * Update pyproject.toml

    * Quick fixes

    * Update llama.py

    * Update llama.py

    * Update dpo.py

    * Update dpo.py

    * Update llama.py

    * Update save.py

    * getattr

commit fc25ab0df0
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Fri Jan 19 22:52:30 2024 +1100

    Quick fixes (#101)

    * Fix tokenizer, dropout, bias for LoRA

    * Update loader.py

    * Fix LoRA downcasting

    * Update _utils.py

    * Saving to GGUF

    * fix

    * colab_quantize_to_gguf

    * move save modules

    * save module

    * Update __init__.py

    * Update save.py

    * Temp downgrade due to TRL issue

    * Fix up bugs

    * Faster saving + other changes

    * Update llama.py

    * Saving modules

    * spelling

    * Update llama.py

    * Update save.py

    * Update save.py

    * Update loader.py

    * Update llama.py

    * patch saving

    * Update save.py

    * Update save.py

    * Update save.py

    * patch saving

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * original_model

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * saving to RAM leakage?

    * Update save.py

    * new_save_directory

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update pyproject.toml

    * Update pyproject.toml

    * Update pyproject.toml

    * Quick fixes

    * Update llama.py

    * Update llama.py

    * Update dpo.py

    * Update dpo.py

    * Update llama.py

    * Update save.py

commit b8b1eafda3
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Fri Jan 19 04:51:19 2024 +1100

    2024 Release (#96)

    * Fix tokenizer, dropout, bias for LoRA

    * Update loader.py

    * Fix LoRA downcasting

    * Update _utils.py

    * Saving to GGUF

    * fix

    * colab_quantize_to_gguf

    * move save modules

    * save module

    * Update __init__.py

    * Update save.py

    * Temp downgrade due to TRL issue

    * Fix up bugs

    * Faster saving + other changes

    * Update llama.py

    * Saving modules

    * spelling

    * Update llama.py

    * Update save.py

    * Update save.py

    * Update loader.py

    * Update llama.py

    * patch saving

    * Update save.py

    * Update save.py

    * Update save.py

    * patch saving

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * original_model

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * saving to RAM leakage?

    * Update save.py

    * new_save_directory

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update pyproject.toml

    * Update pyproject.toml

    * Update pyproject.toml

commit 4112eb4a3d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 19 03:41:00 2024 +1100

    Update pyproject.toml

commit 59d7475336
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 19 03:35:17 2024 +1100

    Update pyproject.toml

commit c1ac4d2707
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Thu Jan 11 04:08:03 2024 +1100

    Fix some bugs (#83)

    * Fix tokenizer, dropout, bias for LoRA

    * Update loader.py

    * Fix LoRA downcasting

    * Update _utils.py

    * Saving to GGUF

    * fix

    * colab_quantize_to_gguf

    * move save modules

    * save module

    * Update __init__.py

    * Update save.py

    * Temp downgrade due to TRL issue

    * Fix up bugs

commit d3887c7fd9
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Wed Jan 10 23:10:48 2024 +1100

    Update README.md (#81)

commit b5d94d9a0a
Author: shimmy <107991372+shimmyshimmer@users.noreply.github.com>
Date:   Wed Jan 10 23:10:23 2024 +1100

    Discord button redo (#80)

commit 01d7f58e11
Author: shimmy <107991372+shimmyshimmer@users.noreply.github.com>
Date:   Wed Jan 10 23:02:20 2024 +1100

    Update logos (#79)

    * HF Perf Button

    * Update README.md

    Adding new buttons cleanup

    * Update README.md

    * Delete images/Discord.png

    * Delete images/try live demo green.png

    * new transparent logos

    * Revamping page

    * Revamp mainpage

    * Update README.md

    * Update README.md

commit 9faaf5b388
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Wed Jan 10 20:03:01 2024 +1100

    Create FUNDING.yml (#78)

commit 82e6fece0b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Jan 10 01:02:44 2024 +1100

    fix_tokenizer

commit b52278199b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 9 23:40:43 2024 +1100

    check_tokenizer

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2024-02-07 02:00:12 +11:00
Daniel Han
d3887c7fd9
Update README.md (#81) 2024-01-10 23:10:48 +11:00
shimmy
01d7f58e11
Update logos (#79)
* HF Perf Button

* Update README.md

Adding new buttons cleanup

* Update README.md

* Delete images/Discord.png

* Delete images/try live demo green.png

* new transparent logos

* Revamping page

* Revamp mainpage

* Update README.md

* Update README.md
2024-01-10 23:02:20 +11:00
Daniel Han
d17202869d
Fix tokenizer, bias, dropout supported for LoRA (#69)
* Fix tokenizer, dropout, bias for LoRA

* Update loader.py
2024-01-06 19:13:39 +11:00
Daniel Han
3fc1305d75
Update README.md (#65) 2024-01-06 04:14:38 +11:00
Daniel Han
f1a4849bb7
Update README.md (#64) 2024-01-06 04:13:15 +11:00
Daniel Han
b7cc0f119f
Update README.md (#63) 2024-01-05 17:48:21 +11:00
Daniel Han
40f384c723
Fix tokenizer + docs (#62)
* Patch tokenizer

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Cleanup

* Add comments to functions

* Update rope_embedding.py

* Update rope_embedding.py

* Update llama.py

* New logos!

* Update README.md
2024-01-05 04:08:53 +11:00
Daniel Han-Chen
a3e4f47ebf Add tokenizer checking + TinyLlama 2024-01-02 03:20:11 +11:00
Daniel Han-Chen
033a897af3 Update README.md 2023-12-31 19:26:34 +11:00
Daniel Han-Chen
c6a0e07029 Update DPO readme 2023-12-31 19:24:30 +11:00
Daniel Han-Chen
0a3477993a DPO 2023-12-31 17:57:45 +11:00
Daniel Han-Chen
74826077da Fix Kaggle issues 2023-12-31 01:20:04 +11:00
Daniel Han
0369e7aa7a
Nightly (#56)
* Pytorch 2.1.1 install path, 4bit loading

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update loader.py

* Update loader.py

* Update loader.py

* Update loader.py

* Spelling errors

* Update __init__.py
2023-12-28 04:19:12 +11:00