ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1,307 commits 391 branches 58 tags 304 MiB
Commit graph

1,307 commits

This branch
This branch
All branches
Author SHA1 Message Date
Daniel Han
0bf5e2be15 Flash Attention whls 2025-05-26 22:48:46 -07:00
Datta Nimmaturi
16a007a283 Upgrade trl fix (#2544) 
* Update llama.py making set and reset functions in order to properly use autoSequenceClassification

* Update fast_lora.py, added mixed precising pytorch autocasting

* Update llama.py did not included rotary embeddings in the reset functions correctly

* Update rl.py: correct get reward model added as well as the eval step stuff

* Update rl.py removed function that did not need to be patched

* Update llama.py: kept reset functions and made their names generic

* Update fast_lora.py

* Update rl.py, try except

* Update fast_lora.py, removing downcasting stuff

* Update llama.py removed depircate LLamaLinearScalingRotaryEmbedding

* Update rl.py for VLLM RLOO and PPO

* Update rl.py reverted

* Update rl.py with peft cahnges

* Update rl.py, disabling adapters screws inference up

* Update rl.py getting PPO support

* Update rl.py cleanup

* Update rl.py cleaned up not useful commented code

* Update llama.py, enabled new flag, keep padding

* Upgrade trl fix

Signed-off-by: Dattu Sharma <venkatadattasainimmaturi@gmail.com>

* Update rl.py made changes relative to the review

* Revert accidental patch block for non grpo

Signed-off-by: Dattu Sharma <venkatadattasainimmaturi@gmail.com>

* Fixup sampling params issue

* Fix rl.py regex

Signed-off-by: Dattu Sharma <venkatadattasainimmaturi@gmail.com>

* loss type: grpo, drgrpo and bnpo

Signed-off-by: Dattu Sharma <venkatadattasainimmaturi@gmail.com>

* Add trl version check for vllm colocate mode for RL trainers

* Update rl.py

For TRL 0.18.0 (Main branch of TRL at the time because its on 0.17.0) , the SFT trainer for some reason deletes the labels column and unsloth internal loss funcitons need that column for hte claculations so I add it back in like this.

* Update llama.py, merge it to be dattas llama version

* Update rl.py, sft changes to get 0.18.0 to be working

* Update rl_replacements.py, added hidden state stuff

* Update rl_replacements.py

* Update rl_replacements.py

* Update rl_replacements.py, rechanged the accumlated loss

* Fixup num_iterations>1 for grpo

Signed-off-by: datta0 <venkatadattasainimmaturi@gmail.com>

* Update rl_replacements.py

* no unnecessary logits upcast. fix naming

Signed-off-by: datta0 <venkatadattasainimmaturi@gmail.com>

* Update rl_replacements.py returned hidden states from logprobs

* Update rl_replacements.py removed debug logic

* Update rl_replacements.py, should be fine now

* Update rl_replacements.py, should take new args for GRPO trainer

* Update rl_replacements.py, made it compatible with trl 0.15.2

* Update rl_replacements.py, fixed typo in per tokne-Logps

---------

Signed-off-by: Dattu Sharma <venkatadattasainimmaturi@gmail.com>
Signed-off-by: datta0 <venkatadattasainimmaturi@gmail.com>
Co-authored-by: pluesclues <136766175+pluesclues@users.noreply.github.com>
 2025-05-26 17:20:57 -07:00
Daniel Han
7a3df703a0 Colocate vLLM 2025-05-26 00:37:04 -07:00
Michael Han
2b419ce039 Update README.md 2025-05-25 03:35:43 -07:00
Quentin Gallouédec
a09f93c921 Remove dataset_text_field from SFTConfig (#2609) 2025-05-25 03:20:16 -07:00
Richi
86c77d40bb add: path checking for failed llama cpp builds (#2603) 2025-05-25 03:18:07 -07:00
Daniel Han
911a1def95 Devstral, MedGemma 2025-05-21 07:35:36 -07:00
Michael Han
a4bb68027e Update README.md 
Updating model support
 2025-05-20 09:51:55 -07:00
Michael Han
ce45ec2b74 Update README.md 2025-05-19 21:26:19 -07:00
Daniel Han
7b57957181 Update issue templates 2025-05-17 18:30:17 -07:00
Daniel Han
2fdfbb09f4 Update issue templates 2025-05-17 18:29:27 -07:00
Daniel Han
17bbb4106e Update issue templates 2025-05-17 05:42:10 -07:00
Daniel Han
1729c7e141 Fix Whisper, ModernBERT (#2565) 
* Update vision.py

* Update vision.py

* Update mapper.py

* Update vision.py

* fix: config.torch_dtype in LlamaModel_fast_forward_inference (#2091)

* fix: config.torch_dtype in LlamaModel_fast_forward_inference

* Update llama.py

* update for consistency

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* versioning

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* model_type_arch

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update loader.py

* check

* Update _utils.py

* Update loader.py

* Update loader.py

* Remove prints

* Update README.md

typo

* Update _utils.py

* Update _utils.py

* versioning

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update vision.py

* HF Transfer

* fix(utils): add missing importlib import to fix NameError (#2134)

This commit fixes a NameError that occurs when `importlib` is referenced in _utils.py
without being imported, especially when UNSLOTH_USE_MODELSCOPE=1 is enabled.
By adding the missing import statement, the code will no longer throw a NameError.

* Add QLoRA Train and Merge16bit Test (#2130)

* add reference and unsloth lora merging tests

* add test / dataset printing to test scripts

* allow running tests from repo root

* add qlora test readme

* more readme edits

* ruff formatting

* additional readme comments

* forgot to add actual tests

* add apache license

* Update pyproject.toml

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update loader.py

* Update loader.py

* Revert

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Bug fix

* Update mapper.py

* check SDPA for Mistral 3, Pixtral

* Update vision.py

* Versioning

* Update rl_replacements.py

* Update README.md

* add model registry

* move hf hub utils to unsloth/utils

* refactor global model info dicts to dataclasses

* fix dataclass init

* fix llama registration

* remove deprecated key function

* start registry reog

* add llama vision

* quant types -> Enum

* remap literal quant types to QuantType Enum

* add llama model registration

* fix quant tag mapping

* add qwen2.5 models to registry

* add option to include original model in registry

* handle quant types per model size

* separate registration of base and instruct llama3.2

* add QwenQVQ to registry

* add gemma3 to registry

* add phi

* add deepseek v3

* add deepseek r1 base

* add deepseek r1 zero

* add deepseek distill llama

* add deepseek distill models

* remove redundant code when constructing model names

* add mistral small to registry

* rename model registration methods

* rename deepseek registration methods

* refactor naming for mistral and phi

* add global register models

* refactor model registration tests for new registry apis

* add model search method

* remove deprecated registration api

* add quant type test

* add registry readme

* make llama registration more specific

* clear registry when executing individual model registration file

* more registry readme updates

* Update _auto_install.py

* Llama4

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Synthetic data

* Update mapper.py

* Xet and Synthetic

* Update synthetic.py

* Update loader.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update pyproject.toml

* Delete .gitignore

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update _utils.py

* Update pyproject.toml

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update chat_templates.py

* Seasame force float16 / float32

* Fix Seasame

* Update loader.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update loader.py

* is_multimodal

* Update loader.py

* Update loader.py

* Update loader.py

* Update loader.py

* Update vision.py

* Update vision.py

* Update vision.py

* UNSLOTH_DISABLE_STATIC_GENERATION

* Update vision.py

* Auto vision detection

* Sesame

* Whisper

* Update loader.py

* Update loader.py

* Update loader.py

* Update mapper.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update loader.py

* Update loader.py

* Update loader.py

* Update loader.py

* Update _utils.py

---------

Co-authored-by: lurf21 <93976703+lurf21@users.noreply.github.com>
Co-authored-by: Jack Shi Wei Lun <87535974+jackswl@users.noreply.github.com>
Co-authored-by: naliazheli <nalia0316@gmail.com>
Co-authored-by: jeromeku <jerome.ku@gmail.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
 2025-05-17 05:11:50 -07:00
Emmanuel Ferdman
ec938fb7e7 Display the model name in RoPE scaling unsupported error (#2564) 
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
 2025-05-17 05:09:06 -07:00
Michael Han
0d221c6300 Merge pull request #2563 from davedgd/main 
fix issue with qwen3 template double quote escapes
 2025-05-16 22:38:39 -07:00
David Dobolyi
1789961183 fix issue with qwen3 template double quote escapes 2025-05-16 23:26:03 -06:00
Etherll
78c9f31c74 Fix trust remote code (#2357) 
* Update _utils.py

* Update loader.py

* Update loader.py

* Update vision.py

* Update unsloth/models/vision.py

* Update unsloth/models/vision.py

* Update unsloth/models/vision.py

* Update unsloth/models/vision.py

* Update unsloth/models/_utils.py

* Update unsloth/models/vision.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
 2025-05-16 16:06:42 -07:00
Daniel Han
8d7c4e13c1 Update pyproject.toml 2025-05-16 15:38:19 -07:00
Daniel Han
7f31d54dc0 Merge branch 'main' of https://github.com/unslothai/unsloth 2025-05-16 15:34:41 -07:00
Daniel Han
953bbdb778 Update _utils.py 2025-05-16 15:33:49 -07:00
Michael Han
0d73478808 Merge pull request #2554 from Erland366/fix/generation_config 
Quick fix on the CompileConfig error
 2025-05-16 12:48:00 -07:00
Erland366
9e9f2a6229 Fix Nonetype on the compile_config 2025-05-16 13:16:34 +00:00
Michael Han
84ab0b2fd0 Update README.md 2025-05-16 01:56:40 -07:00
Michael Han
2ccadeb474 Update README.md 
TTS support
 2025-05-15 15:15:53 -07:00
Daniel Han
532a43f828 TTS (#2545) 
* Update rl_replacements.py

* Update vision.py

* Update rl_replacements.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Remove double generate patch

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update mapper.py

* Update vision.py

* fix: config.torch_dtype in LlamaModel_fast_forward_inference (#2091)

* fix: config.torch_dtype in LlamaModel_fast_forward_inference

* Update llama.py

* update for consistency

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* versioning

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* model_type_arch

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update loader.py

* check

* Update _utils.py

* Update loader.py

* Update loader.py

* Remove prints

* Update README.md

typo

* Update _utils.py

* Update _utils.py

* versioning

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update vision.py

* HF Transfer

* fix(utils): add missing importlib import to fix NameError (#2134)

This commit fixes a NameError that occurs when `importlib` is referenced in _utils.py
without being imported, especially when UNSLOTH_USE_MODELSCOPE=1 is enabled.
By adding the missing import statement, the code will no longer throw a NameError.

* Add QLoRA Train and Merge16bit Test (#2130)

* add reference and unsloth lora merging tests

* add test / dataset printing to test scripts

* allow running tests from repo root

* add qlora test readme

* more readme edits

* ruff formatting

* additional readme comments

* forgot to add actual tests

* add apache license

* Update pyproject.toml

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update loader.py

* Update loader.py

* Revert

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Bug fix

* Update mapper.py

* check SDPA for Mistral 3, Pixtral

* Update vision.py

* Versioning

* Update rl_replacements.py

* Update README.md

* add model registry

* move hf hub utils to unsloth/utils

* refactor global model info dicts to dataclasses

* fix dataclass init

* fix llama registration

* remove deprecated key function

* start registry reog

* add llama vision

* quant types -> Enum

* remap literal quant types to QuantType Enum

* add llama model registration

* fix quant tag mapping

* add qwen2.5 models to registry

* add option to include original model in registry

* handle quant types per model size

* separate registration of base and instruct llama3.2

* add QwenQVQ to registry

* add gemma3 to registry

* add phi

* add deepseek v3

* add deepseek r1 base

* add deepseek r1 zero

* add deepseek distill llama

* add deepseek distill models

* remove redundant code when constructing model names

* add mistral small to registry

* rename model registration methods

* rename deepseek registration methods

* refactor naming for mistral and phi

* add global register models

* refactor model registration tests for new registry apis

* add model search method

* remove deprecated registration api

* add quant type test

* add registry readme

* make llama registration more specific

* clear registry when executing individual model registration file

* more registry readme updates

* Update _auto_install.py

* Llama4

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Synthetic data

* Update mapper.py

* Xet and Synthetic

* Update synthetic.py

* Update loader.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update pyproject.toml

* Delete .gitignore

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update _utils.py

* Update pyproject.toml

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update chat_templates.py

* Seasame force float16 / float32

* Fix Seasame

* Update loader.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update loader.py

* is_multimodal

* Update loader.py

* Update loader.py

* Update loader.py

* Update loader.py

* Update vision.py

* Update vision.py

* Update vision.py

* UNSLOTH_DISABLE_STATIC_GENERATION

* Update vision.py

* Auto vision detection

* Sesame

* Whisper

* Update loader.py

* Update loader.py

* Update loader.py

---------

Co-authored-by: lurf21 <93976703+lurf21@users.noreply.github.com>
Co-authored-by: Jack Shi Wei Lun <87535974+jackswl@users.noreply.github.com>
Co-authored-by: naliazheli <nalia0316@gmail.com>
Co-authored-by: jeromeku <jerome.ku@gmail.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
 2025-05-15 09:23:52 -07:00
Michael Han
f30b16b0d4 Update README.md 2025-05-15 06:54:05 -07:00
Janusz
22f2fea2ea Add use_rslora reference to LoraConfig inititalisation (#2539) 
Co-authored-by: jkumz <janusz.kumor01@gmail.com>
 2025-05-15 04:24:18 -07:00
omahs
568a806fc8 Fix typos (#2540) 2025-05-15 04:23:27 -07:00
Michael Han
e01f304c6d Merge pull request #2537 from kiankyars/main 
Add Qwen-3 chat template and Ollama template support
 2025-05-14 20:45:53 -07:00
Kian Kyars
4e7b91f11d undo accident 2025-05-14 19:00:23 -06:00
Kian Kyars
973a0ddcd4 style: Place Qwen-3 template after Gemma-3, match style with other templates 2025-05-14 18:59:17 -06:00
Kian Kyars
d3fdd4193b Update Qwen-3 chat and Ollama templates to official full version, placed after Gemma-3 2025-05-14 18:42:55 -06:00
Kian Kyars
31887228d6 Add Qwen-3 chat template and Ollama template support 2025-05-14 18:35:02 -06:00
Daniel Han
4170900b47 Update pyproject.toml 2025-05-14 05:42:47 -07:00
Daniel Han
75cdebf9b7 Update _utils.py 2025-05-14 05:42:11 -07:00
Daniel Han
957c203f0e Update synthetic.py 2025-05-14 04:05:23 -07:00
Daniel Han
62e669b291 Update synthetic.py 2025-05-14 03:54:46 -07:00
Daniel Han
9dd46b5a6d Update synthetic.py 2025-05-14 03:49:27 -07:00
Daniel Han
175c39e9b1 Update synthetic.py 2025-05-14 03:47:29 -07:00
Daniel Han
d6fbd3043e Update synthetic.py 2025-05-14 03:44:15 -07:00
Michael Han
7d608fef58 Merge pull request #2527 from mmathew23/csm 
Add Sesame CSM
 2025-05-14 02:26:12 -07:00
DoubleMathew
89448d22a7 Merge branch 'unslothai:main' into csm 2025-05-13 17:58:11 -05:00
Daniel Han
77ebc7b74e Versioning 2025-05-13 09:10:24 -07:00
Michael Han
99e12bcc78 Update README.md 2025-05-13 01:39:59 -07:00
Daniel Han
1b71e6d542 Update loader_utils.py 2025-05-12 21:06:30 -07:00
Daniel Han
c0ac1cdb01 Update pyproject.toml 2025-05-12 16:28:50 -07:00
feng lui
c9ca2096b7 vLLM Windows CUDA support [tested] (#2158) 
* Update loader.py

change vllm installed check by transformers utils function

* Update llama.py

change vllm installed check by transformers utils function

* add sample notebook

* fix Indentation

* add global is_vLLM_available function

* Pythonic style

* Delete nb/Qwen2.5_(3B)-GRPO-windows.ipynb

Would be great to move it to https://github.com/unslothai/notebooks - appreciate it!

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
 2025-05-12 05:33:42 -07:00
Daniel Han
32ba81f076 Update pyproject.toml 2025-05-12 04:24:18 -07:00
Daniel Han
ac5fc0bc46 Fix Intel GPU 2025-05-12 03:10:21 -07:00
Lei Zhenyuan
6d61258dd3 [2/N] Enable intel GPU for unsloth (#2388) 
* add DEVICE_TYPE and resolve device specific API

* reuse import torch

* move env under device typr

* resolve comments

* add more comments

* add more comments
 2025-05-12 02:58:21 -07:00