Michael Han
|
e6270d3790
|
Update README.md
Unsloth Dynamic 4-bit Quantization Update
|
2024-12-04 21:32:23 -08:00 |
|
Michael Han
|
8de7843e21
|
Update README.md
Fixing Qwen links
|
2024-12-03 16:50:52 -08:00 |
|
Daniel Han
|
d30c363d25
|
Vision (#1318)
* Add files via upload
* Add files via upload
* Add files via upload
* Add files via upload
* Update README.md
* Update README.md
* Update README.md
* Update README.md
---------
Co-authored-by: Michael <107991372+shimmyshimmer@users.noreply.github.com>
|
2024-11-21 11:24:12 -08:00 |
|
Daniel Han
|
0de5457252
|
Bug fixes (#1288)
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165)
* chore: update chat_templates.py (#1166)
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
* Update _utils.py
* fix/transformers-unpack (#1180)
* Fix DPO, ORPO (#1177)
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165)
* chore: update chat_templates.py (#1166)
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* donot upcast lm_head and embeddings to float32 (#1186)
* Cleanup upcast logs (#1188)
* Fix/phi-longrope (#1193)
* Enhance rotary embedding handling in LlamaAttention and LongRopeRotaryEmbedding
* Typo
* Improve rotary embedding handling in LlamaAttention to prevent errors with short KV cache
* Update llama.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update transformers
* Unk token issues
* Update _utils.py
* Fix pad token
* Update llama.py
* Typo
* ignored labels
* Revert "ignored labels"
This reverts commit 9d07be077b.
* More patching
* Update _utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Feat/all tmp (#1219)
* Update save.py
Check whether path is in /tmp dir for Kaggle environment
* Update save.py
Move temporary_location to /tmp in Kaggle
* Enhance Kaggle environment support in save and tokenizer utilities
---------
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
* Bug fixes
* Update pyproject.toml
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Tied weights
* Revert "Tied weights"
This reverts commit 8090b7c01a.
* Tied weights
* Utils
* CE Loss patching
* Update __init__.py
* Update __init__.py
* Patching
* Update cross_entropy_loss.py
* CE Loss
* Update _utils.py
* Update _utils.py
* CE Loss
* Update _utils.py
* Update _utils.py
* Layernorm
* Update _utils.py
* Update _utils.py
* Post patch
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* typing
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* int64
* Update _utils.py
* Update cross_entropy_loss.py
* constexpr
* constexpr
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* CE
* Update cross_entropy_loss.py
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update utils.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* typing
* Update rope_embedding.py
* types
* Disable compiling
* Update _utils.py
* Update _utils.py
* Forward hook
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update llama.py
* Update llama.py
* Update _utils.py
* Update pyproject.toml
* Update _utils.py
* Update llama.py
* CE Loss
* Update cross_entropy_loss.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update llama.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Fix: cast logits to float32 in cross_entropy_forward to prevent errors (#1254)
* Fix: cast logits to float32 in cross_entropy_forward to prevent errors
* Update cross_entropy_loss.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Throw error when inferencing longer than max_popsition_embeddings (#1236)
* Throw error when inferencing longer than max_popsition_embeddings without rope scaling
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* CLI now handles user input strings for dtype correctly (#1235)
Co-authored-by: root <root@ieeres.chu.cam.ac.uk>
* Update flex_attention.py
* Update _utils.py
* Update _utils.py
* Update flex_attention.py
* Update flex_attention.py
* Update loader.py
* Update loader.py
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update _utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* triton_cast
* Update utils.py
* Qwen 2.5 Coder
* Fix/export mistral (#1281)
* Enhance install_python_non_blocking to handle protobuf installation and process management
* Revert "Enhance install_python_non_blocking to handle protobuf installation and process management"
This reverts commit f09974b151.
* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266
* Revert "Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266"
This reverts commit 9fc130785d.
* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266
* Update __init__.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* DOC Update - Update README.md with os.environ in example (#1269)
* Update README.md with os.environ in example
Added OS Environ in example to avoid device conflicts , for a user at least in jupyter notebook this allows to select GPU in a multi GPU setup.
As currently the unsloth init checks all GPU's and takes the first in the order which can be a issue when some GPU's are in use and the list still shows them. So to manually avoid this, this os config is required.
Small change but a bit time saver for those who straight away copies the tutorials
* Update README.md
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix/get_chat_template (#1246)
* Refactor `get_chat_template` to now support system message instead. It supposed to fix ollama tokenizer chattemplate to
* Remove type hinting
* Update chat_templates.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix/sft-trainer (#1276)
* Add patch for SFTTrainer to maintain backward compatibility with TRL changes
* Update trainer.py
* Update trainer.py
* Refactor trainer patch to maintain backward compatibility with TRL changes
* Update trainer.py
* Refactor trainer.py to exclude non-convertible trainers from backward compatibility patch
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update __init__.py
* Update trainer.py
* Update trainer.py
* Update trainer.py
* Update tokenizer_utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
Co-authored-by: Edwin Fennell <edwinfennell1@gmail.com>
Co-authored-by: root <root@ieeres.chu.cam.ac.uk>
Co-authored-by: Uday Girish Maradana <einsteingirish@gmail.com>
|
2024-11-13 19:05:40 -08:00 |
|
Daniel Han
|
8d46c0d4d6
|
Torch 2.5
|
2024-10-26 18:03:15 -07:00 |
|
Daniel Han
|
0e5a507f87
|
Many bug fixes (#1162)
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165)
* chore: update chat_templates.py (#1166)
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
|
2024-10-23 03:14:57 -07:00 |
|
Daniel Han
|
b3e85e915e
|
Update README.md
|
2024-10-17 20:46:11 -07:00 |
|
Daniel Han
|
eb533dbf14
|
Update README.md
|
2024-10-17 20:45:40 -07:00 |
|
Daniel Han
|
39bff2f034
|
Update README.md
|
2024-10-01 00:40:17 -07:00 |
|
Daniel Han
|
7c8f227681
|
Update README.md
|
2024-09-26 00:12:42 -07:00 |
|
Daniel Han
|
2462e5c51b
|
Update README.md
|
2024-09-26 00:05:38 -07:00 |
|
Daniel Han
|
b0f3dc087a
|
Update README.md
|
2024-09-26 00:02:15 -07:00 |
|
Daniel Han
|
3ad8309b00
|
Update README.md
|
2024-09-23 01:36:50 -07:00 |
|
Daniel Han
|
1038c723bc
|
Qwen 2.5
|
2024-09-23 01:27:12 -07:00 |
|
Daniel Han
|
8aceff3e7b
|
Update README.md (#1036)
|
2024-09-18 13:23:45 -07:00 |
|
Daniel Han
|
575c1bd67d
|
Update README.md (#1033)
|
2024-09-15 17:42:09 -07:00 |
|
Daniel Han
|
7476d4b5f6
|
Update README.md
|
2024-09-08 14:30:54 -07:00 |
|
Daniel Han
|
d674f1c852
|
Update README.md
|
2024-09-08 12:29:31 -07:00 |
|
Daniel Han
|
20506463e1
|
Phi 3.5 bug fix (#955)
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* update token retrieval logic (#952)
* Fix DPO (#947)
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* update hf token retrieval logic
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update llama.py
* get_token
* Update README.md
---------
Co-authored-by: Hafedh <70411813+not-lain@users.noreply.github.com>
|
2024-08-23 17:38:24 -07:00 |
|
Daniel Han
|
600ffe2a17
|
Update README.md (#941)
Co-authored-by: Michael <107991372+shimmyshimmer@users.noreply.github.com>
|
2024-08-20 17:59:50 -07:00 |
|
Daniel Han
|
d5768379c5
|
Phi 3.5 (#940)
* LongRoPE
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update mapper.py
* Phi 3.5
|
2024-08-20 16:51:39 -07:00 |
|
Daniel Han
|
d45ade257b
|
Update README.md (#938)
|
2024-08-19 17:18:30 -07:00 |
|
Daniel Han
|
bfe38e6ea8
|
Update README.md
|
2024-08-05 00:00:53 -07:00 |
|
Daniel Han
|
b43855fb36
|
Update README.md
|
2024-08-04 23:59:57 -07:00 |
|
Daniel Han
|
291bc6e254
|
Update README.md
|
2024-08-04 23:50:40 -07:00 |
|
emuchogu
|
46b4348698
|
pascal support (#870)
Co-authored-by: Edward Muchogu <muchogu@gmail.com>
|
2024-08-04 23:45:51 -07:00 |
|
Daniel Han
|
dfca5516e7
|
Update README.md
|
2024-07-31 09:50:11 -07:00 |
|
Daniel Han
|
b85670de83
|
Gemma (#843)
* bugs
* Update _utils.py
* flash-attn softcapping
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update mapper.py
* Update README.md
* Update _utils.py
|
2024-07-31 08:54:58 -07:00 |
|
Daniel Han
|
a96d16e44b
|
Update README.md
|
2024-07-23 15:08:09 -07:00 |
|
Daniel Han
|
824511e265
|
Update README.md
|
2024-07-23 12:07:27 -07:00 |
|
Daniel Han
|
22968a2134
|
Update README.md
|
2024-07-23 11:51:08 -07:00 |
|
Daniel Han
|
c86b13d465
|
Llama 3.1 (#797)
* Llama 3.1
* Update _utils.py
* Llama 3.1
* Update _utils.py
* Update llama.py
* Update llama.py
* hack for rotary
* patch RoPE
* refix rope
* Update _utils.py
* Update llama.py
* Llama 3.1 check
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
|
2024-07-23 11:40:49 -07:00 |
|
Daniel Han
|
568bfddfc3
|
Update README.md
|
2024-07-19 03:05:15 -07:00 |
|
Daniel Han
|
39885a4ddc
|
Update README.md
|
2024-07-19 03:03:50 -07:00 |
|
Daniel Han
|
5ab565fb2c
|
Gemma2 (#723)
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
---------
Co-authored-by: Michael <107991372+shimmyshimmer@users.noreply.github.com>
|
2024-07-03 12:12:21 -07:00 |
|
Daniel Han
|
b34a44e969
|
Qwen bug fixes (#639)
* Update llama.py
* offload
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* continued pretraining trainer
* Update trainer.py
* Update trainer.py
* Update trainer.py
* Update trainer.py
* is_bfloat16_supported
* Update __init__.py
* Update README.md
* Update llama.py
* is_bfloat16_supported
* Update __init__.py
* Mistral v3
* Phi 3 medium
* Update chat_templates.py
* Update chat_templates.py
* Phi-3
* Update save.py
* Update README.md
Mistral v3 to Mistral v0.3
* Untrained tokens
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update llama.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update save.py
* Update save.py
* Update save.py
* checkpoint
* Update _utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update llama.py
* accelerate
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update tokenizer_utils.py
* train_dataloader
* Update llama.py
* Update llama.py
* Update llama.py
* use_fast_convert
* Update save.py
* Update save.py
* Update save.py
* Update save.py
* remove_special_tokens
* Ollama
* Update chat_templates.py
* Update chat_templates.py
* Update chat_templates.py
* Update llama.py
* Update chat_templates.py
* Support bfloat16 GGUF
* Update save.py
* Update llama.py
* fast_forward_inference
* Update mapper.py
* Update loader.py
* Update llama.py
* Update tokenizer_utils.py
* info
* edits
* Create chat template
* Fix tokenizer
* Update tokenizer_utils.py
* fix case where gguf saving fails due to first_conversion dtype (#630)
* Support revision parameter in FastLanguageModel.from_pretrained (#629)
* support `revision` parameter
* match unsloth formatting of named parameters
* clears any selected_adapters before calling internal_model.save_pretrained (#609)
* Update __init__.py (#602)
Check for incompatible modules before importing unsloth
* Fixed unsloth/tokenizer_utils.py for chat training (#604)
* Add GGML saving option to Unsloth for easier Ollama model creation and testing. (#345)
* Add save to llama.cpp GGML to save.py.
* Fix conversion command and path of convert to GGML function.
* Add autosaving lora to the GGML function
* Create lora save function for conversion to GGML
* Test fix #2 for saving lora
* Test fix #3 to save the lora adapters to convert to GGML
* Remove unwated tokenizer saving for conversion to ggml and added a few print statements.
* Needed tokenizer for saving, added it back, also made it more unslothy style by having positional arguments, and added a few messages.
* Positional arguments didn't work out, so reverted to older version of the code, and added a few comments.
* Test fix 1 for arch
* Test fix 2 new Mistral error.
* Test fix 3
* Revert to old version for testing.
* Upload issue test fix 1
* Fix 2 uploading ggml
* Positional ags added.
* Temporray remove positional args
* Fix upload again!!!
* Add print statements and fix link
* Make the calling name better
* Create local saving for GGML
* Add choosing directory to save local GGML.
* Fix lil variable error in the save_to_custom_dir func
* docs: Add LoraConfig parameters documentation (#619)
* llama.cpp failing (#371)
llama.cpp is failing to generate quantize versions for the trained models.
Error:
```bash
You might have to compile llama.cpp yourself, then run this again.
You do not need to close this Python program. Run the following commands in a new terminal:
You must run this in the same folder as you're saving your model.
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make clean && LLAMA_CUDA=1 make all -j
Once that's done, redo the quantization.
```
But when i do clone this with recursive it works.
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix libcuda_dirs import for triton 3.0 (#227)
* fix libcuda_dirs import for triton 3.0
* Update __init__.py
* Update __init__.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update save.py
* Update __init__.py
* Update fast_lora.py
* Update save.py
* Update save.py
* Update save.py
* Update loader.py
* Update save.py
* Update save.py
* quantize now llama-quantize
* Update chat_templates.py
* Update loader.py
* Update mapper.py
* Update __init__.py
* embedding size
* Update qwen2.py
* docs
* Update README.md
* Update qwen2.py
* README: Fix minor typo. (#559)
* README: Fix minor typo.
One-character typo fix while reading.
* Update README.md
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update mistral.py
* Update qwen2.py
* Update qwen2.py
* Update qwen2.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update README.md
* FastMistralModel
---------
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Eliot Hall <60240707+chrehall68@users.noreply.github.com>
Co-authored-by: Rickard Edén <rickardeden@gmail.com>
Co-authored-by: XiaoYang <xyangk@gmail.com>
Co-authored-by: Oseltamivir <58582368+Oseltamivir@users.noreply.github.com>
Co-authored-by: mahiatlinux <110882203+mahiatlinux@users.noreply.github.com>
Co-authored-by: Sébastien De Greef <sebdg@binarycompany.com>
Co-authored-by: Alberto Ferrer <albertof@barrahome.org>
Co-authored-by: Thomas Viehmann <tv.github-private@beamnet.de>
Co-authored-by: Walter Korman <lemurware@gmail.com>
|
2024-06-14 20:59:45 +10:00 |
|
Daniel Han-Chen
|
83807103ca
|
Update README.md
|
2024-06-05 06:15:27 +10:00 |
|
Daniel Han-Chen
|
f9689b1472
|
Update README.md
|
2024-06-05 06:14:11 +10:00 |
|
Daniel Han
|
9515038867
|
Nightly (#548)
* Update llama.py
* offload
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* continued pretraining trainer
* Update trainer.py
* Update trainer.py
* Update trainer.py
* Update trainer.py
* is_bfloat16_supported
* Update __init__.py
* Update README.md
* Update llama.py
* is_bfloat16_supported
* Update __init__.py
* Mistral v3
* Phi 3 medium
* Update chat_templates.py
* Update chat_templates.py
* Phi-3
* Update save.py
* Update README.md
Mistral v3 to Mistral v0.3
* Untrained tokens
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update llama.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update save.py
* Update save.py
* Update save.py
* checkpoint
---------
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
|
2024-05-29 00:30:38 +10:00 |
|
Daniel Han
|
2e1cb3888b
|
Phi-3, Llama-3 bug fixes (#519)
* Update llama.py
* offload
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* continued pretraining trainer
* Update trainer.py
* Update trainer.py
* Update trainer.py
* Update trainer.py
* is_bfloat16_supported
* Update __init__.py
* Update README.md
* Update llama.py
* is_bfloat16_supported
* Update __init__.py
* Mistral v3
* Phi 3 medium
* Update chat_templates.py
* Update chat_templates.py
* Phi-3
|
2024-05-24 06:36:10 +10:00 |
|
Daniel Han-Chen
|
a106e0b256
|
Update README.md
|
2024-05-23 04:31:26 +10:00 |
|
Daniel Han
|
5134a42f06
|
Nightly (#506)
* Update llama.py
* offload
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* continued pretraining trainer
* Update trainer.py
* Update trainer.py
* Update trainer.py
* Update trainer.py
* is_bfloat16_supported
* Update __init__.py
* Update README.md
* Update llama.py
|
2024-05-22 04:45:57 +10:00 |
|
Daniel Han
|
d4512f7c13
|
May 2024 Prelim (#447)
* Fix prompt
* Update chat_templates.py
* fix_untrained_tokens
* Update llama.py
* add tokens
* Update _utils.py
* Update tokenizer_utils.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* pad_token
* Update chat_templates.py
* Update chat_templates.py
* tokenizer
* Update save.py
* Update chat_templates.py
* Update chat_templates.py
* patch tokenizer padding
* Update tokenizer_utils.py
* Update save.py
* Fix: loading models with resized vocabulary (#377)
* new: vocab resize on load
* new: gitignore
* GGUF fix
* Readme (#390)
* Update README.md
* Update README.md
---------
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
* Update README.md
* Delete .gitignore
* Phi-3
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Fix reserved tokens
* Update save.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update chat_templates.py
* Update save.py
* Update _utils.py
* Update chat_templates.py
* Adds dependencies and extras for torch 2.3.0 with new xformers versions (#415)
* Adds dependencies and extras for torch 2.3.0 with new xformers versions
* Add 2.3.0 section to readme
* Support Qwen2 (#428)
* support Qwen2
* support Qwen2
* Delete README.md
* Revert "Delete README.md"
This reverts commit 026b05f859.
* Update README.md
* Qwen2 == Mistral
* Update llama.py
* Update __init__.py
* Update README.md
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update save.py
* Update save.py
* Update _utils.py
* Update save.py
* Update save.py
* Update save.py
* test_hf_gguf_equivalence
* Update chat_templates.py
* Update chat_templates.py
* --pad-vocab
* Update tokenizer_utils.py
---------
Co-authored-by: Igor Kilbas <whitemarsstudios@gmail.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Nathan Azrak <42650258+nathan-az@users.noreply.github.com>
Co-authored-by: Yang JianXin <995462226@qq.com>
|
2024-05-13 05:22:03 +10:00 |
|
Daniel Han-Chen
|
72aee4a3a1
|
Update README.md
|
2024-04-30 20:26:10 +10:00 |
|
Daniel Han
|
81819c337d
|
Phi-3 (#397)
* Fix prompt
* Update chat_templates.py
* fix_untrained_tokens
* Update llama.py
* add tokens
* Update _utils.py
* Update tokenizer_utils.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* pad_token
* Update chat_templates.py
* Update chat_templates.py
* tokenizer
* Update save.py
* Update chat_templates.py
* Update chat_templates.py
* patch tokenizer padding
* Update tokenizer_utils.py
* Update save.py
* Fix: loading models with resized vocabulary (#377)
* new: vocab resize on load
* new: gitignore
* GGUF fix
* Readme (#390)
* Update README.md
* Update README.md
---------
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
* Update README.md
* Delete .gitignore
* Phi-3
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
---------
Co-authored-by: Igor Kilbas <whitemarsstudios@gmail.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
|
2024-04-30 05:59:02 +10:00 |
|
Daniel Han
|
5039ea3c36
|
Nightly (#370)
* Fix prompt
* Update chat_templates.py
* fix_untrained_tokens
* Update llama.py
* add tokens
* Update _utils.py
* Update tokenizer_utils.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* pad_token
* Update chat_templates.py
* Update chat_templates.py
* tokenizer
* Update save.py
* Update chat_templates.py
* Update chat_templates.py
* patch tokenizer padding
* Update tokenizer_utils.py
* Update save.py
* Fix: loading models with resized vocabulary (#377)
* new: vocab resize on load
* new: gitignore
* GGUF fix
* Readme (#390)
* Update README.md
* Update README.md
---------
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
* Update README.md
* Delete .gitignore
---------
Co-authored-by: Igor Kilbas <whitemarsstudios@gmail.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
|
2024-04-29 04:47:03 +10:00 |
|
Daniel Han-Chen
|
ce523ed15c
|
Update README.md
|
2024-04-20 14:22:52 +10:00 |
|
Daniel Han
|
88eee50267
|
Update README.md (#352)
|
2024-04-19 05:50:19 +10:00 |
|
Daniel Han
|
9c7bbecb50
|
Update README.md (#351)
|
2024-04-19 05:47:04 +10:00 |
|
Daniel Han
|
4606443b77
|
Readme Changes (#324)
* Update README.md
* Update README.md
---------
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
|
2024-04-11 01:43:34 +10:00 |
|