ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
99 commits 391 branches 58 tags 304 MiB
Commit graph

99 commits

This branch
This branch
All branches
Author SHA1 Message Date
Daniel Han
45fa4bf356 Fix tokenizer, bias, dropout supported for LoRA (#69) 
* Fix tokenizer, dropout, bias for LoRA

* Update loader.py
 2024-01-06 19:13:39 +11:00
Daniel Han
e4a080b68e Update README.md (#65) 2024-01-06 04:14:38 +11:00
Daniel Han
f077568c6b Update README.md (#64) 2024-01-06 04:13:15 +11:00
Daniel Han
905bd58f7f Update README.md (#63) 2024-01-05 17:48:21 +11:00
Daniel Han
353432271d Fix tokenizer + docs (#62) 
* Patch tokenizer

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Cleanup

* Add comments to functions

* Update rope_embedding.py

* Update rope_embedding.py

* Update llama.py

* New logos!

* Update README.md
 2024-01-05 04:08:53 +11:00
Daniel Han-Chen
808211931d Update llama.py 2024-01-02 03:41:05 +11:00
Daniel Han-Chen
9a09d161f2 Update llama.py 2024-01-02 03:37:04 +11:00
Daniel Han-Chen
ca4a51048a Add tokenizer checking + TinyLlama 2024-01-02 03:20:11 +11:00
Daniel Han-Chen
9406ca6ec0 Update README.md 2023-12-31 19:26:34 +11:00
Daniel Han-Chen
5fbe143340 Update DPO readme 2023-12-31 19:24:30 +11:00
Daniel Han-Chen
148e7db6c3 PatchDPOTrainer 2023-12-31 18:36:43 +11:00
Daniel Han-Chen
473554810a DPO 2023-12-31 18:26:17 +11:00
Daniel Han-Chen
1665b06884 DPO 2023-12-31 17:57:45 +11:00
Daniel Han-Chen
3c3d4974ce Fix Kaggle issues 2023-12-31 01:20:04 +11:00
Daniel Han
aab090d20f Prelim Qwen, Deepseek support (#58) 
* Pytorch 2.1.1 install path, 4bit loading

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update loader.py

* Update loader.py

* Update loader.py

* Update loader.py

* Spelling errors

* Update __init__.py

* DPO loss fix

* Update fast_lora.py

* Update fast_lora.py

* Out of bounds tokenization

* Fix Mistral SWA

* Prelim support Qwen, Deepseek etc
 2023-12-30 04:35:35 +11:00
Daniel Han
e18736d5b2 DPO, SWA fixes (#57) 
* Pytorch 2.1.1 install path, 4bit loading

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update loader.py

* Update loader.py

* Update loader.py

* Update loader.py

* Spelling errors

* Update __init__.py

* DPO loss fix

* Update fast_lora.py

* Update fast_lora.py

* Out of bounds tokenization

* Fix Mistral SWA
 2023-12-29 04:55:01 +11:00
Daniel Han
28c7550571 Nightly (#56) 
* Pytorch 2.1.1 install path, 4bit loading

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update loader.py

* Update loader.py

* Update loader.py

* Update loader.py

* Spelling errors

* Update __init__.py
 2023-12-28 04:19:12 +11:00
Daniel Han-Chen
56fe2cf49d Fix inference 2023-12-27 03:51:31 +11:00
Daniel Han-Chen
75d1a0e2e1 Fix FastLanguageModel 2023-12-27 03:01:45 +11:00
Daniel Han
04093395ed Fix RoPE Scaling issues (#52) 
* Fix RoPE Scaling

* Update llama.py

* Update llama.py
 2023-12-26 04:32:04 +11:00
Daniel Han-Chen
5dda3f4058 Fix Pytorch 2.1.1 2023-12-25 23:28:05 +11:00
Daniel Han
ef70177a24 Small fixes (#48) 
* Fix generation for GQA

* Update _utils.py

* flash attn

* Update _utils.py

* Update llama.py

* Update mistral.py

* platform

* Update _utils.py

* Update llama.py

* Logo changed

* Update README.md

* Update README.md
 2023-12-23 04:22:48 +11:00
Daniel Han
37365b6ba9 Fix typo 2023-12-18 12:27:31 +11:00
shimmy
51d6797c7c Update __init__.py 
Fixing typo extra '='
 2023-12-18 12:19:24 +11:00
Daniel Han-Chen
feec338c08 Torch version, docs, readme, general loader 2023-12-18 04:23:16 +11:00
Daniel Han-Chen
cf5204a52d Fix Mistral 
BlockDiagonalCausalMask fix courtesy of https://github.com/Rypo
 2023-12-16 13:14:33 +11:00
Daniel Han-Chen
44bf41e113 Update Colab to CUDA 12.1 2023-12-15 16:35:24 +11:00
Daniel Han-Chen
9c6803a573 Fix packaging, readme 2023-12-15 12:57:08 +11:00
Daniel Han
4143ca52e1 Update README.md 2023-12-14 15:10:17 +11:00
Daniel Han
d6fdf151a5 Update README.md 2023-12-14 12:33:51 +11:00
Daniel Han-Chen
9714bf7438 Update llama.py 2023-12-12 13:07:38 +11:00
Daniel Han-Chen
0e0e03c239 Update llama.py 2023-12-12 02:54:54 +11:00
Daniel Han-Chen
b5a9c9c69d tokenizer pad fix 2023-12-12 02:40:37 +11:00
Daniel Han-Chen
870ca65421 Update mistral.py 2023-12-12 00:59:30 +11:00
Daniel Han-Chen
3c790a6680 Update llama.py 2023-12-12 00:57:06 +11:00
Daniel Han-Chen
0912303bfb Mistral, GQA support 2023-12-11 23:58:36 +11:00
Daniel Han-Chen
1fa07c58fb Fix generation 2023-12-07 00:13:29 +11:00
Daniel Han
6416f235cf Pre-release 2023 December version (Mistral, Prelim DPO, WSL, bug fixes) (#16) 
* Immediate bug fixes

* Update README.md

* Update README.md

* Update llama.py

* Update llama.py

* Rope Scaling and max_seq_len will change

* Update llama.py

* new images

* Update README.md

* Images

* Update README.md

* Update pyproject.toml

* GQA

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py
 2023-12-06 02:59:22 +11:00
Daniel Han-Chen
d1c1748266 Sticker 2023-12-02 03:08:16 +11:00
Daniel Han
30c792ee58 Update README.md 2023-12-01 21:54:31 +11:00
Daniel Han
c1ed527ebe Update README.md 2023-12-01 19:54:04 +11:00
Daniel Han
9f65d942ff Update README.md 2023-12-01 17:50:40 +11:00
Daniel Han
ab9ac86aa0 Update README.md 2023-12-01 16:07:53 +11:00
Daniel Han
a636326598 Update README.md 2023-12-01 16:07:38 +11:00
Daniel Han-Chen
77c527b15d Create Discord.png 2023-12-01 16:04:15 +11:00
Daniel Han
0de488b302 Update README.md 2023-12-01 14:42:52 +11:00
Daniel Han
8712684094 Update README.md 2023-12-01 14:42:42 +11:00
Daniel Han
30932a7831 Update README.md 2023-12-01 14:38:03 +11:00
Daniel Han
0cfb1fa0ad Update README.md 2023-12-01 14:35:51 +11:00
Daniel Han
7178c5d3a8 Update README.md 2023-12-01 14:11:49 +11:00