Daniel Han
|
f2ff798c4e
|
Xformers
|
2024-12-31 22:42:28 -08:00 |
|
Daniel Han
|
10a7d4fc93
|
Update pyproject.toml
|
2024-12-29 19:34:30 -08:00 |
|
Daniel Han
|
4b9c6110b1
|
Update pyproject.toml
|
2024-12-29 03:57:21 -08:00 |
|
Daniel Han
|
da4741bbef
|
Update pyproject.toml
|
2024-12-26 04:12:46 -08:00 |
|
Daniel Han
|
47f210922d
|
Update pyproject.toml
|
2024-12-26 04:04:23 -08:00 |
|
Daniel Han
|
30bb143a87
|
Update pyproject.toml
|
2024-12-26 03:26:01 -08:00 |
|
Daniel Han
|
258d8135e7
|
Update pyproject.toml
|
2024-12-24 03:35:04 -08:00 |
|
Daniel Han
|
fd2295d6a6
|
Update pyproject.toml
|
2024-12-24 01:55:04 -08:00 |
|
Daniel Han
|
77ca6e2dca
|
Triton windows
|
2024-12-24 00:17:33 -08:00 |
|
Daniel Han
|
f97f49b08d
|
Update pyproject.toml
|
2024-12-24 00:08:49 -08:00 |
|
Daniel Han
|
660df7b9ab
|
Update pyproject.toml
|
2024-12-24 00:08:03 -08:00 |
|
Daniel Han
|
61147bfbad
|
Update pyproject.toml
|
2024-12-24 00:07:37 -08:00 |
|
Daniel Han
|
6e6e65bd46
|
Update pyproject.toml
|
2024-12-24 00:02:30 -08:00 |
|
Daniel Han
|
e6d50c8839
|
Update pyproject.toml
|
2024-12-23 22:49:16 -08:00 |
|
Daniel Han
|
5323157160
|
Update pyproject.toml
|
2024-12-23 22:47:20 -08:00 |
|
Daniel Han
|
fba4cf7fb4
|
Update pyproject.toml
|
2024-12-23 22:47:11 -08:00 |
|
Daniel Han
|
753b954702
|
dependencies
|
2024-12-23 22:27:35 -08:00 |
|
Daniel Han
|
c939334570
|
Update pyproject.toml
|
2024-12-23 21:53:43 -08:00 |
|
Daniel Han
|
01b256b4cf
|
Update pyproject.toml
|
2024-12-23 21:52:50 -08:00 |
|
Daniel Han
|
4cc97b23cc
|
Version
|
2024-12-23 21:43:32 -08:00 |
|
Daniel Han
|
2f969fa137
|
Update pyproject.toml
|
2024-12-23 21:16:30 -08:00 |
|
Daniel Han
|
25c5b0524d
|
Update pyproject.toml
|
2024-12-23 21:14:01 -08:00 |
|
Daniel Han
|
69dc1ad694
|
Version
|
2024-12-23 21:07:51 -08:00 |
|
Daniel Han
|
e8ae4727d0
|
Update pyproject.toml
|
2024-12-21 01:24:09 -08:00 |
|
Daniel Han
|
58bcf66d27
|
Temp fix
|
2024-12-20 03:07:04 -08:00 |
|
Daniel Han
|
8144766f78
|
Update pyproject.toml
|
2024-11-26 03:29:59 -08:00 |
|
Daniel Han
|
4fe74c93ee
|
Update pyproject.toml
|
2024-11-21 17:46:22 -08:00 |
|
Daniel Han
|
15268ba184
|
Bug fixes (#1245)
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165)
* chore: update chat_templates.py (#1166)
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
* Update _utils.py
* fix/transformers-unpack (#1180)
* Fix DPO, ORPO (#1177)
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165)
* chore: update chat_templates.py (#1166)
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* donot upcast lm_head and embeddings to float32 (#1186)
* Cleanup upcast logs (#1188)
* Fix/phi-longrope (#1193)
* Enhance rotary embedding handling in LlamaAttention and LongRopeRotaryEmbedding
* Typo
* Improve rotary embedding handling in LlamaAttention to prevent errors with short KV cache
* Update llama.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update transformers
* Unk token issues
* Update _utils.py
* Fix pad token
* Update llama.py
* Typo
* ignored labels
* Revert "ignored labels"
This reverts commit 9d07be077b.
* More patching
* Update _utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Feat/all tmp (#1219)
* Update save.py
Check whether path is in /tmp dir for Kaggle environment
* Update save.py
Move temporary_location to /tmp in Kaggle
* Enhance Kaggle environment support in save and tokenizer utilities
---------
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
* Bug fixes
* Update pyproject.toml
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Tied weights
* Revert "Tied weights"
This reverts commit 8090b7c01a.
* Tied weights
* Utils
* CE Loss patching
* Update __init__.py
* Update __init__.py
* Patching
* Update cross_entropy_loss.py
* CE Loss
* Update _utils.py
* Update _utils.py
* CE Loss
* Update _utils.py
* Update _utils.py
* Layernorm
* Update _utils.py
* Update _utils.py
* Post patch
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* typing
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* int64
* Update _utils.py
* Update cross_entropy_loss.py
* constexpr
* constexpr
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* CE
* Update cross_entropy_loss.py
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update utils.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* typing
* Update rope_embedding.py
* types
* Disable compiling
* Update _utils.py
* Update _utils.py
* Forward hook
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update llama.py
* Update llama.py
* Update _utils.py
* Update pyproject.toml
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
|
2024-11-05 13:29:37 -08:00 |
|
Daniel Han
|
9d5f58224d
|
Update pyproject.toml
|
2024-10-26 18:05:55 -07:00 |
|
Daniel Han
|
e7ede2f7db
|
Torch 2.5
|
2024-10-26 18:03:15 -07:00 |
|
Daniel Han
|
4c85177719
|
Many bug fixes (#1162)
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165)
* chore: update chat_templates.py (#1166)
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
|
2024-10-23 03:14:57 -07:00 |
|
Daniel Han
|
d2a032e117
|
Gradient Accumulation Fix (#1134)
* Unsloth Zoo
* Update trainer.py
* Update trainer.py
* Update cross_entropy_loss.py
* n_items
* Update llama.py
* kwargs
* Remove extraneous f prefixes (#1133)
Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>
* Update __init__.py
---------
Co-authored-by: Emil Sadek <esadek@hotmail.com>
Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>
|
2024-10-14 19:17:35 -07:00 |
|
Daniel Han
|
e130e748f0
|
Reload
|
2024-10-05 17:21:48 -07:00 |
|
Daniel Han
|
248c27d205
|
Fix merges (#1079)
* Layernorm
* Update layernorm.py
* Update layernorm.py
* Update layernorm.py
* Update layernorm.py
* Update layernorm.py
* Update layernorm.py
* Patch layernorm
* Update layernorm.py
* RMS Layernorm
* Update rms_layernorm.py
* Causal LM
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update layernorm.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* Llama 3.2
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update llama.py
* Update vision.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update loader.py
* Update loader.py
* Update loader.py
* Dependencies
* Update pyproject.toml
* Update _utils.py
|
2024-09-30 03:03:01 -07:00 |
|
Daniel Han
|
b314837622
|
Update pyproject.toml
|
2024-09-27 01:36:45 -07:00 |
|
Daniel Han
|
930d2ad1a8
|
Update pyproject.toml
|
2024-09-25 23:47:15 -07:00 |
|
Daniel Han
|
5b345ec757
|
Update pyproject.toml
|
2024-09-25 23:13:49 -07:00 |
|
Daniel Han
|
1397f2e1ab
|
Torch 2.4, Xformers>0.0.27, TRL>0.9, Python 3.12 + bug fixes (#902)
* Update pyproject.toml
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update _utils.py
* Update _utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* fix_tokenizer
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update save.py
* Update save.py
* Update save.py
* Update save.py
* Update save.py
* Update loader.py
* Update pyproject.toml
* Update _utils.py
* Update gemma2.py
* Update gemma2.py
* Update _utils.py
* gemma 2 mask
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update _utils.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update _utils.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Torch 2.4 Xformers 0.0.27post2
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Gemma 2 fixes
* Update gemma2.py
* Update llama.py
* Update llama.py
* Update save.py
* Update save.py
|
2024-08-10 19:59:40 -07:00 |
|
Daniel Han
|
734e478605
|
Fix tokenizers (#887)
* Update pyproject.toml
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update _utils.py
* Update _utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* fix_tokenizer
* Update tokenizer_utils.py
* Update tokenizer_utils.py
|
2024-08-06 20:24:44 -07:00 |
|
Daniel Han
|
9283909b6d
|
Update pyproject.toml
|
2024-08-04 11:28:21 -07:00 |
|
Daniel Han
|
282d8a5794
|
flash-attn softcapping
|
2024-07-30 22:48:53 -07:00 |
|
Daniel Han
|
e6159e0279
|
Checks
|
2024-07-27 22:16:33 -07:00 |
|
Daniel Han
|
d28527ce62
|
Edits
|
2024-07-27 20:30:32 -07:00 |
|
Daniel Han
|
a2403852b4
|
Llama 3.1 check
|
2024-07-23 11:09:24 -07:00 |
|
Daniel Han
|
66ce2d401a
|
Update pyproject.toml
|
2024-07-18 10:59:09 -07:00 |
|
Daniel Han
|
f176cbd36a
|
Many bug fixes (#754)
* Update gemma2.py
* Update llama.py
* Update llama.py
* Update gemma2.py
* init
* Update gemma2.py
* Update gemma2.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* All RoPE Scaling support
* cleanup
* Update llama.py
* Update llama.py
* Update _utils.py
* Update _utils.py
* exec
* exec
* Attention_Module
* attention_module
* imports
* exec
* Update llama.py
* Update llama.py
* boolean mask
* revert masking
* Update llama.py
* Update save.py
* Update llama.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update utils.py
* retry
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update _utils.py
* Update _utils.py
* Update gemma2.py
* Update chat_templates.py
* Gemma 2 Ollama support
* Update llama.py
* Update llama.py
* error handling
* Update _utils.py
* Update _utils.py
* Stats for debugging
* Update _utils.py
* Update _utils.py
* Debugging
* Update tokenizer_utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Check exec, eval
* Update _utils.py
* Update _utils.py
* Images
* Bug fixes
* Update pyproject.toml
* Bug fixes
* Update _utils.py
* Update _utils.py
|
2024-07-10 01:59:06 -07:00 |
|
Daniel Han
|
cc4c5d7785
|
Gemma2 (#709)
* Update mapper.py
* Update loader.py
* Update llama.py
* Update tokenizer_utils.py
* info
* edits
* Create chat template
* Fix tokenizer
* Update tokenizer_utils.py
* fix case where gguf saving fails due to first_conversion dtype (#630)
* Support revision parameter in FastLanguageModel.from_pretrained (#629)
* support `revision` parameter
* match unsloth formatting of named parameters
* clears any selected_adapters before calling internal_model.save_pretrained (#609)
* Update __init__.py (#602)
Check for incompatible modules before importing unsloth
* Fixed unsloth/tokenizer_utils.py for chat training (#604)
* Add GGML saving option to Unsloth for easier Ollama model creation and testing. (#345)
* Add save to llama.cpp GGML to save.py.
* Fix conversion command and path of convert to GGML function.
* Add autosaving lora to the GGML function
* Create lora save function for conversion to GGML
* Test fix #2 for saving lora
* Test fix #3 to save the lora adapters to convert to GGML
* Remove unwated tokenizer saving for conversion to ggml and added a few print statements.
* Needed tokenizer for saving, added it back, also made it more unslothy style by having positional arguments, and added a few messages.
* Positional arguments didn't work out, so reverted to older version of the code, and added a few comments.
* Test fix 1 for arch
* Test fix 2 new Mistral error.
* Test fix 3
* Revert to old version for testing.
* Upload issue test fix 1
* Fix 2 uploading ggml
* Positional ags added.
* Temporray remove positional args
* Fix upload again!!!
* Add print statements and fix link
* Make the calling name better
* Create local saving for GGML
* Add choosing directory to save local GGML.
* Fix lil variable error in the save_to_custom_dir func
* docs: Add LoraConfig parameters documentation (#619)
* llama.cpp failing (#371)
llama.cpp is failing to generate quantize versions for the trained models.
Error:
```bash
You might have to compile llama.cpp yourself, then run this again.
You do not need to close this Python program. Run the following commands in a new terminal:
You must run this in the same folder as you're saving your model.
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make clean && LLAMA_CUDA=1 make all -j
Once that's done, redo the quantization.
```
But when i do clone this with recursive it works.
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix libcuda_dirs import for triton 3.0 (#227)
* fix libcuda_dirs import for triton 3.0
* Update __init__.py
* Update __init__.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update save.py
* Update __init__.py
* Update fast_lora.py
* Update save.py
* Update save.py
* Update save.py
* Update loader.py
* Update save.py
* Update save.py
* quantize now llama-quantize
* Update chat_templates.py
* Update loader.py
* Update mapper.py
* Update __init__.py
* embedding size
* Update qwen2.py
* docs
* Update README.md
* Update qwen2.py
* README: Fix minor typo. (#559)
* README: Fix minor typo.
One-character typo fix while reading.
* Update README.md
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update mistral.py
* Update qwen2.py
* Update qwen2.py
* Update qwen2.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update README.md
* FastMistralModel
* Update mistral.py
* Update mistral.py
* Update mistral.py
* Update mistral.py
* Update mistral.py
* Auto check rope scaling
* Update llama.py
* Update llama.py
* Update llama.py
* GPU support
* Typo
* Update gemma.py
* gpu
* Multiple GGUF saving
* Update save.py
* Update save.py
* check PEFT and base
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update chat_templates.py
* Fix breaking bug in save.py with interpreting quantization_method as a string when saving to gguf (#651)
* Nightly (#649)
* Update llama.py
* offload
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* continued pretraining trainer
* Update trainer.py
* Update trainer.py
* Update trainer.py
* Update trainer.py
* is_bfloat16_supported
* Update __init__.py
* Update README.md
* Update llama.py
* is_bfloat16_supported
* Update __init__.py
* Mistral v3
* Phi 3 medium
* Update chat_templates.py
* Update chat_templates.py
* Phi-3
* Update save.py
* Update README.md
Mistral v3 to Mistral v0.3
* Untrained tokens
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update llama.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update save.py
* Update save.py
* Update save.py
* checkpoint
* Update _utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update llama.py
* accelerate
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update tokenizer_utils.py
* train_dataloader
* Update llama.py
* Update llama.py
* Update llama.py
* use_fast_convert
* Update save.py
* Update save.py
* Update save.py
* Update save.py
* remove_special_tokens
* Ollama
* Update chat_templates.py
* Update chat_templates.py
* Update chat_templates.py
* Update llama.py
* Update chat_templates.py
* Support bfloat16 GGUF
* Update save.py
* Update llama.py
* fast_forward_inference
* Update mapper.py
* Update loader.py
* Update llama.py
* Update tokenizer_utils.py
* info
* edits
* Create chat template
* Fix tokenizer
* Update tokenizer_utils.py
* fix case where gguf saving fails due to first_conversion dtype (#630)
* Support revision parameter in FastLanguageModel.from_pretrained (#629)
* support `revision` parameter
* match unsloth formatting of named parameters
* clears any selected_adapters before calling internal_model.save_pretrained (#609)
* Update __init__.py (#602)
Check for incompatible modules before importing unsloth
* Fixed unsloth/tokenizer_utils.py for chat training (#604)
* Add GGML saving option to Unsloth for easier Ollama model creation and testing. (#345)
* Add save to llama.cpp GGML to save.py.
* Fix conversion command and path of convert to GGML function.
* Add autosaving lora to the GGML function
* Create lora save function for conversion to GGML
* Test fix #2 for saving lora
* Test fix #3 to save the lora adapters to convert to GGML
* Remove unwated tokenizer saving for conversion to ggml and added a few print statements.
* Needed tokenizer for saving, added it back, also made it more unslothy style by having positional arguments, and added a few messages.
* Positional arguments didn't work out, so reverted to older version of the code, and added a few comments.
* Test fix 1 for arch
* Test fix 2 new Mistral error.
* Test fix 3
* Revert to old version for testing.
* Upload issue test fix 1
* Fix 2 uploading ggml
* Positional ags added.
* Temporray remove positional args
* Fix upload again!!!
* Add print statements and fix link
* Make the calling name better
* Create local saving for GGML
* Add choosing directory to save local GGML.
* Fix lil variable error in the save_to_custom_dir func
* docs: Add LoraConfig parameters documentation (#619)
* llama.cpp failing (#371)
llama.cpp is failing to generate quantize versions for the trained models.
Error:
```bash
You might have to compile llama.cpp yourself, then run this again.
You do not need to close this Python program. Run the following commands in a new terminal:
You must run this in the same folder as you're saving your model.
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make clean && LLAMA_CUDA=1 make all -j
Once that's done, redo the quantization.
```
But when i do clone this with recursive it works.
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix libcuda_dirs import for triton 3.0 (#227)
* fix libcuda_dirs import for triton 3.0
* Update __init__.py
* Update __init__.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update save.py
* Update __init__.py
* Update fast_lora.py
* Update save.py
* Update save.py
* Update save.py
* Update loader.py
* Update save.py
* Update save.py
* quantize now llama-quantize
* Update chat_templates.py
* Update loader.py
* Update mapper.py
* Update __init__.py
* embedding size
* Update qwen2.py
* docs
* Update README.md
* Update qwen2.py
* README: Fix minor typo. (#559)
* README: Fix minor typo.
One-character typo fix while reading.
* Update README.md
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update mistral.py
* Update qwen2.py
* Update qwen2.py
* Update qwen2.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update README.md
* FastMistralModel
* Update mistral.py
* Update mistral.py
* Update mistral.py
* Update mistral.py
* Update mistral.py
* Auto check rope scaling
* Update llama.py
* Update llama.py
* Update llama.py
* GPU support
* Typo
* Update gemma.py
* gpu
* Multiple GGUF saving
* Update save.py
* Update save.py
* check PEFT and base
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update chat_templates.py
---------
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Eliot Hall <60240707+chrehall68@users.noreply.github.com>
Co-authored-by: Rickard Edén <rickardeden@gmail.com>
Co-authored-by: XiaoYang <xyangk@gmail.com>
Co-authored-by: Oseltamivir <58582368+Oseltamivir@users.noreply.github.com>
Co-authored-by: mahiatlinux <110882203+mahiatlinux@users.noreply.github.com>
Co-authored-by: Sébastien De Greef <sebdg@binarycompany.com>
Co-authored-by: Alberto Ferrer <albertof@barrahome.org>
Co-authored-by: Thomas Viehmann <tv.github-private@beamnet.de>
Co-authored-by: Walter Korman <lemurware@gmail.com>
* Fix bug in save.py with interpreting quantization_method as a string that prevents GGUF from saving
* Implemented better list management and then forgot to actually call the new list variable, fixed
* Check type of given quantization method and return type error if not list or string
* Update save.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Eliot Hall <60240707+chrehall68@users.noreply.github.com>
Co-authored-by: Rickard Edén <rickardeden@gmail.com>
Co-authored-by: XiaoYang <xyangk@gmail.com>
Co-authored-by: Oseltamivir <58582368+Oseltamivir@users.noreply.github.com>
Co-authored-by: mahiatlinux <110882203+mahiatlinux@users.noreply.github.com>
Co-authored-by: Sébastien De Greef <sebdg@binarycompany.com>
Co-authored-by: Alberto Ferrer <albertof@barrahome.org>
Co-authored-by: Thomas Viehmann <tv.github-private@beamnet.de>
Co-authored-by: Walter Korman <lemurware@gmail.com>
* Revert "Fix breaking bug in save.py with interpreting quantization_method as …" (#652)
This reverts commit 506cb68867296237e95bc53c32f1bfc9b1757960.
* Revert "Revert "Fix breaking bug in save.py with interpreting quantization_me…" (#653)
This reverts commit 2f48cc9af385579876fd45bd833169d1f1a2ea58.
* Update llama.py
* peft
* patch
* Update loader.py
* retrain
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* offload
* Update llama.py
* Create a starter script for command-line training to integrate in ML ops pipelines. (#623)
* Update chat_templates.py
* Ollama
* Update chat_templates.py
* Update chat_templates.py
* Update chat_templates.py
* Update chat_templates.py
* Update chat_templates.py
* Update chat_templates.py
* Update chat_templates.py
* Update chat_templates.py
* Update chat_templates.py
* Update chat_templates.py
* Ollama
* Update chat_templates.py
* ollama
* Update mapper.py
* Update chat_templates.py
* Update save.py
* Update save.py
* Update save.py
* Update save.py
* Update save.py
* Update save.py
* Update save.py
* Update chat_templates.py
* Update chat_templates.py
* Update chat_templates.py
* Update chat_templates.py
* Update llama.py
* Fixes
* clearer messages
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update llama.py
* Update llama.py
* Update llama.py
* log
* Update __init__.py
* Update llama.py
* Update __init__.py
* Create Merge.png
* Create ollama.png
* Gemma2
* Update llama.py
* Update loader.py
* Update pyproject.toml
* Update pyproject.toml
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update _utils.py
* Revert Gemma2
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update rms_layernorm.py
* Update gemma2.py
* logit softcapping
* Update cross_entropy_loss.py
* Update llama.py
* Update llama.py
* Update gemma2.py
* Update gemma2.py
* Update cross_entropy_loss.py
* Update llama.py
* Update llama.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update llama.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update llama.py
* Update gemma2.py
* Update llama.py
* Update llama.py
* Update gemma2.py
* Update gemma2.py
* Update llama.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update _utils.py
* Update _utils.py
* Update gemma2.py
* compile flags
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update gemma2.py
* Update gemma2.py
* fixes
* Update _utils.py
* Fix generation
* Update llama.py
* Update llama.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* pad token
* Update gemma2.py
* pad token
* Update _utils.py
* Update llama.py
* Update gemma2.py
* edit warning
* Update tokenizer_utils.py
---------
Co-authored-by: Eliot Hall <60240707+chrehall68@users.noreply.github.com>
Co-authored-by: Rickard Edén <rickardeden@gmail.com>
Co-authored-by: XiaoYang <xyangk@gmail.com>
Co-authored-by: Oseltamivir <58582368+Oseltamivir@users.noreply.github.com>
Co-authored-by: mahiatlinux <110882203+mahiatlinux@users.noreply.github.com>
Co-authored-by: Sébastien De Greef <sebdg@binarycompany.com>
Co-authored-by: Alberto Ferrer <albertof@barrahome.org>
Co-authored-by: Thomas Viehmann <tv.github-private@beamnet.de>
Co-authored-by: Walter Korman <lemurware@gmail.com>
Co-authored-by: ArcadaLabs-Jason <52756218+ArcadaLabs-Jason@users.noreply.github.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
|
2024-07-02 22:51:01 -07:00 |
|
Daniel Han-Chen
|
0451f82140
|
Update pyproject.toml
|
2024-06-06 01:22:29 +10:00 |
|
Daniel Han
|
dde6a0f0d3
|
Nightly (#483)
* peft issue
* Update save.py
* Update __init__.py
* Update pyproject.toml
|
2024-05-17 23:46:33 +10:00 |
|
Daniel Han
|
63e175a77d
|
peft issue (#480)
|
2024-05-17 04:18:18 +10:00 |
|