ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1,287 commits 391 branches 58 tags 304 MiB
Commit graph

1,287 commits

This branch
This branch
All branches
Author SHA1 Message Date
Mathew Mathew
bbe5e2a221 the pixtral vision notebook fails during inferenc with unused kwargs token_type_ids. This fixes the error 2025-05-02 21:40:15 -05:00
Daniel Han
9edbe23259 Bug fix 2025-05-02 14:06:14 -07:00
Daniel Han
d957caeae7 Fix Qwen 3 mapping 2025-05-02 09:44:01 -07:00
Michael Han
8bfe5fd4ab Update README.md 2025-05-02 09:06:57 -07:00
Daniel Han
2249a5ff88 Qwen3 bug fixes 2025-05-02 07:18:33 -07:00
Daniel Han
8fbd80231c Update mapper.py 2025-05-02 06:26:43 -07:00
Daniel Han
d74e7c19b3 Versioning 2025-05-02 05:05:09 -07:00
Daniel Han
439cae9fae Remove hf_xet warning 2025-05-02 04:23:53 -07:00
Daniel Han
f90224b02f Update __init__.py 2025-05-02 03:43:44 -07:00
Daniel Han
2a231c7ba4 Merge branch 'main' of https://github.com/unslothai/unsloth 2025-05-02 03:09:52 -07:00
Daniel Han
eb495f171f Qwen 3 2025-05-02 03:09:44 -07:00
cblomert
629a3fcfe3 Added k_norm & q_norm to merged Qwen3 layers (#2452) 2025-05-02 03:07:37 -07:00
Michael Han
97a63f809f Update README.md 
Qwen3 notebook
 2025-05-01 22:52:42 -07:00
Daniel Han
6e2b5a767f Nightly (#2448) 
* move float32

* Ensure trust_remote_code propegates down to unsloth_compile_transformers (#2075)

* Update _utils.py

* Show both `peft_error` and `autoconfig_error`, not just `autoconfig_error` (#2080)

When loading a PEFT model fails, only the `autoconfig_error` is shown. Instead of the `peft_error`, which is what really matters when we're trying to load a PEFT adapter, the user will see something like this:

```
RuntimeError: Unrecognized model in my_model. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: albert, align, altclip, ...
```

This PR just changes it so `autoconfig_error` and `peft_error` are both displayed.

* fix error message (#2046)

* Update vision.py

* Update _utils.py

* Update pyproject.toml

* Update __init__.py

* Update __init__.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update rl_replacements.py

* Update rl_replacements.py

* Update rl_replacements.py

* Update rl_replacements.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update rl_replacements.py

* Update vision.py

* Update rl_replacements.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Remove double generate patch

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update mapper.py

* Update vision.py

* fix: config.torch_dtype in LlamaModel_fast_forward_inference (#2091)

* fix: config.torch_dtype in LlamaModel_fast_forward_inference

* Update llama.py

* update for consistency

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* versioning

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* model_type_arch

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update loader.py

* check

* Update _utils.py

* Update loader.py

* Update loader.py

* Remove prints

* Update README.md

typo

* Update _utils.py

* Update _utils.py

* versioning

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update vision.py

* HF Transfer

* fix(utils): add missing importlib import to fix NameError (#2134)

This commit fixes a NameError that occurs when `importlib` is referenced in _utils.py
without being imported, especially when UNSLOTH_USE_MODELSCOPE=1 is enabled.
By adding the missing import statement, the code will no longer throw a NameError.

* Add QLoRA Train and Merge16bit Test (#2130)

* add reference and unsloth lora merging tests

* add test / dataset printing to test scripts

* allow running tests from repo root

* add qlora test readme

* more readme edits

* ruff formatting

* additional readme comments

* forgot to add actual tests

* add apache license

* Update pyproject.toml

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update loader.py

* Update loader.py

* Revert

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Bug fix

* Update mapper.py

* check SDPA for Mistral 3, Pixtral

* Update vision.py

* Versioning

* Update rl_replacements.py

* Update README.md

* add model registry

* move hf hub utils to unsloth/utils

* refactor global model info dicts to dataclasses

* fix dataclass init

* fix llama registration

* remove deprecated key function

* start registry reog

* add llama vision

* quant types -> Enum

* remap literal quant types to QuantType Enum

* add llama model registration

* fix quant tag mapping

* add qwen2.5 models to registry

* add option to include original model in registry

* handle quant types per model size

* separate registration of base and instruct llama3.2

* add QwenQVQ to registry

* add gemma3 to registry

* add phi

* add deepseek v3

* add deepseek r1 base

* add deepseek r1 zero

* add deepseek distill llama

* add deepseek distill models

* remove redundant code when constructing model names

* add mistral small to registry

* rename model registration methods

* rename deepseek registration methods

* refactor naming for mistral and phi

* add global register models

* refactor model registration tests for new registry apis

* add model search method

* remove deprecated registration api

* add quant type test

* add registry readme

* make llama registration more specific

* clear registry when executing individual model registration file

* more registry readme updates

* Update _auto_install.py

* Llama4

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Synthetic data

* Update mapper.py

* Xet and Synthetic

* Update synthetic.py

* Update loader.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update pyproject.toml

* Delete .gitignore

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update _utils.py

* Update pyproject.toml

* Update synthetic.py

* Update synthetic.py

---------

Co-authored-by: Xander Hawthorne <167850078+CuppaXanax@users.noreply.github.com>
Co-authored-by: Isaac Breen <isaac.breen@icloud.com>
Co-authored-by: Kareem <81531392+KareemMusleh@users.noreply.github.com>
Co-authored-by: lurf21 <93976703+lurf21@users.noreply.github.com>
Co-authored-by: Jack Shi Wei Lun <87535974+jackswl@users.noreply.github.com>
Co-authored-by: naliazheli <nalia0316@gmail.com>
Co-authored-by: jeromeku <jerome.ku@gmail.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
 2025-05-01 07:42:08 -07:00
Daniel Han
382961c9d3 Update mapper.py 2025-05-01 03:07:56 -07:00
Daniel Han
9a930bb095 Qwen 3, Bug Fixes (#2445) 
* bug fix #2008 (#2039)

* fix (#2051)

* Update loader.py

* Update pyproject.toml

* Update pyproject.toml

* Update vision.py

* more prints

* Update loader.py

* LoRA 16bit fix

* Update vision.py

* Update vision.py

* Update _utils.py

* Update vision.py

* move forced float32

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* move print

* Update _utils.py

* disable bfloat16

* Fix forced float32

* move float32

* Ensure trust_remote_code propegates down to unsloth_compile_transformers (#2075)

* Update _utils.py

* Show both `peft_error` and `autoconfig_error`, not just `autoconfig_error` (#2080)

When loading a PEFT model fails, only the `autoconfig_error` is shown. Instead of the `peft_error`, which is what really matters when we're trying to load a PEFT adapter, the user will see something like this:

```
RuntimeError: Unrecognized model in my_model. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: albert, align, altclip, ...
```

This PR just changes it so `autoconfig_error` and `peft_error` are both displayed.

* fix error message (#2046)

* Update vision.py

* Update _utils.py

* Update pyproject.toml

* Update __init__.py

* Update __init__.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update rl_replacements.py

* Update rl_replacements.py

* Update rl_replacements.py

* Update rl_replacements.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update rl_replacements.py

* Update vision.py

* Update rl_replacements.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Remove double generate patch

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update mapper.py

* Update vision.py

* fix: config.torch_dtype in LlamaModel_fast_forward_inference (#2091)

* fix: config.torch_dtype in LlamaModel_fast_forward_inference

* Update llama.py

* update for consistency

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* versioning

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* model_type_arch

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update loader.py

* check

* Update _utils.py

* Update loader.py

* Update loader.py

* Remove prints

* Update README.md

typo

* Update _utils.py

* Update _utils.py

* versioning

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update vision.py

* HF Transfer

* fix(utils): add missing importlib import to fix NameError (#2134)

This commit fixes a NameError that occurs when `importlib` is referenced in _utils.py
without being imported, especially when UNSLOTH_USE_MODELSCOPE=1 is enabled.
By adding the missing import statement, the code will no longer throw a NameError.

* Add QLoRA Train and Merge16bit Test (#2130)

* add reference and unsloth lora merging tests

* add test / dataset printing to test scripts

* allow running tests from repo root

* add qlora test readme

* more readme edits

* ruff formatting

* additional readme comments

* forgot to add actual tests

* add apache license

* Update pyproject.toml

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update loader.py

* Update loader.py

* Revert

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Bug fix

* Update mapper.py

* check SDPA for Mistral 3, Pixtral

* Update vision.py

* Versioning

* Update rl_replacements.py

* Update README.md

* add model registry

* move hf hub utils to unsloth/utils

* refactor global model info dicts to dataclasses

* fix dataclass init

* fix llama registration

* remove deprecated key function

* start registry reog

* add llama vision

* quant types -> Enum

* remap literal quant types to QuantType Enum

* add llama model registration

* fix quant tag mapping

* add qwen2.5 models to registry

* add option to include original model in registry

* handle quant types per model size

* separate registration of base and instruct llama3.2

* add QwenQVQ to registry

* add gemma3 to registry

* add phi

* add deepseek v3

* add deepseek r1 base

* add deepseek r1 zero

* add deepseek distill llama

* add deepseek distill models

* remove redundant code when constructing model names

* add mistral small to registry

* rename model registration methods

* rename deepseek registration methods

* refactor naming for mistral and phi

* add global register models

* refactor model registration tests for new registry apis

* add model search method

* remove deprecated registration api

* add quant type test

* add registry readme

* make llama registration more specific

* clear registry when executing individual model registration file

* more registry readme updates

* Update _auto_install.py

* Llama4

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Synthetic data

* Update mapper.py

* Xet and Synthetic

* Update synthetic.py

* Update loader.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update synthetic.py

* Update pyproject.toml

* Delete .gitignore

---------

Co-authored-by: Mukkesh Ganesh <mukmckenzie@gmail.com>
Co-authored-by: Kareem <81531392+KareemMusleh@users.noreply.github.com>
Co-authored-by: Xander Hawthorne <167850078+CuppaXanax@users.noreply.github.com>
Co-authored-by: Isaac Breen <isaac.breen@icloud.com>
Co-authored-by: lurf21 <93976703+lurf21@users.noreply.github.com>
Co-authored-by: Jack Shi Wei Lun <87535974+jackswl@users.noreply.github.com>
Co-authored-by: naliazheli <nalia0316@gmail.com>
Co-authored-by: jeromeku <jerome.ku@gmail.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
 2025-04-30 22:38:39 -07:00
Daniel Han
599a15ea8d Update pyproject.toml 2025-04-30 22:34:07 -07:00
Daniel Han
43d483122f Merge branch 'main' of https://github.com/unslothai/unsloth 2025-04-30 07:31:34 -07:00
Michael Han
f4283a800e Merge pull request #2439 from Etherll/patch-1 
Update mapper.py to add Qwen3 base
 2025-04-30 06:18:46 -07:00
Etherll
d4ef475c33 Update mapper.py 2025-04-30 14:39:23 +03:00
Daniel Han
a7eb02790e Update synthetic.py 2025-04-29 23:48:33 -07:00
Daniel Han
f03961da09 Update synthetic.py 2025-04-29 23:47:09 -07:00
Michael Han
af337af35c Merge pull request #2436 from Datta0/qwen3_support 
Qwen3 inference fixes
 2025-04-29 20:18:03 -07:00
Dattu Sharma
5640401435 Qwen3 inference fixes 2025-04-30 03:03:38 +00:00
Daniel Han
ecfd56aabe Update _utils.py 2025-04-29 11:43:24 -07:00
Daniel Han
7b97cf2304 Update synthetic.py 2025-04-29 11:43:13 -07:00
Daniel Han
7e6dbd9ccd Create __init__.py 2025-04-29 11:18:03 -07:00
Daniel Han
8babbeaded Create synthetic.py 2025-04-29 11:17:10 -07:00
Daniel Han
9b5446d5a2 Versioning 2025-04-29 09:50:51 -07:00
Daniel Han
d3f419f6ac Update mapper.py 2025-04-29 00:50:46 -07:00
Michael Han
9945bcb629 Merge pull request #2427 from Datta0/qwen3_support 
Fixup qwen3 qk norm
 2025-04-28 23:45:21 -07:00
Dattu Sharma
8cb2400e45 fixup qwen3 qk norm 
Signed-off-by: Dattu Sharma <venkatadattasainimmaturi@gmail.com>
 2025-04-29 06:22:07 +00:00
Michael Han
f637694085 Merge pull request #2423 from Datta0/qwen3_support 
Fixup qwen3
 2025-04-28 19:32:45 -07:00
Dattu Sharma
26cf059574 fixup qwen3 
Signed-off-by: Dattu Sharma <venkatadattasainimmaturi@gmail.com>
 2025-04-29 02:31:55 +00:00
Michael Han
53e6fba362 Update README.md 2025-04-28 19:08:12 -07:00
Michael Han
e4ab75296d Merge pull request #2211 from Datta0/qwen3_support 
[WIP] Initial support for Qwen3. Will udpate when the model is released
 2025-04-28 19:05:40 -07:00
Daniel Han
acea2c09bb Update pyproject.toml 2025-04-26 03:45:29 -07:00
Daniel Han
de64444965 Versioning 2025-04-26 03:44:21 -07:00
Michael Han
31a37cb2ce Merge pull request #2381 from Erland366/fix/saving_vlm_4bit 
Fix saving 4bit for VLM
 2025-04-19 14:26:11 -07:00
Erland366
ed16a50bf9 feat: Add validation for 4bit save method and implement corresponding error handling 2025-04-19 20:36:30 +00:00
Michael Han
f1128eea1b Merge pull request #2375 from unslothai/revert-2358-patch-1 
Revert "fix: improved error handling when llama.cpp build fails"
 2025-04-17 20:33:40 -07:00
Michael Han
6b261264a2 Revert "fix: improved error handling when llama.cpp build fails" 2025-04-17 20:33:25 -07:00
Michael Han
7a5b81db74 Merge pull request #2358 from Hansehart/patch-1 
fix: improved error handling when llama.cpp build fails
 2025-04-17 14:03:45 -07:00
Richi
4b3ae022a9 fix: improved error handling when llama.cpp build fails 2025-04-16 09:17:25 +02:00
Etherll
9fc1e3b945 feat: Support custom auto_model for wider model compatibility (Whisper, Bert,etc) & attn_implementation support (#2263) 
* Update loader.py

* Update vision.py

* Update vision.py

fix attn_implementation

* Refactor: Improve parameter handling and checks in loader/vision
 2025-04-14 14:10:05 -07:00
Michael Han
a41cda7dfb Update question.md 2025-04-09 15:31:56 -07:00
Michael Han
76f24eb8a2 Update feature_request.md 2025-04-09 15:30:56 -07:00
Michael Han
ea8b427fc2 Update documentation.md 2025-04-09 15:29:20 -07:00
Michael Han
3b10fbde0a Update bug_report.md 2025-04-09 15:24:35 -07:00
Michael Han
92c87a4e97 Update question.md 2025-04-09 15:20:44 -07:00