Daniel Han
db292872c8
Update fp8.py
2025-11-25 06:50:58 -08:00
Daniel Han
cb00618e49
Float8
2025-11-25 06:48:10 -08:00
Daniel Han
fee3841c24
Update fp8.py
2025-11-25 05:35:34 -08:00
Daniel Han
c69a5ffe8e
Fix auto installer
2025-11-25 01:47:58 -08:00
Daniel Han
b3612aa636
Add torch 2.9.1
2025-11-25 01:36:11 -08:00
Daniel Han
bfb57e564c
Update rl.py
2025-11-24 22:13:17 -08:00
Daniel Han
6f8fe138a5
Update vision.py
2025-11-24 05:50:22 -08:00
Daniel Han
a04af0d884
Merge branch 'main' into nightly
2025-11-24 02:16:53 -08:00
Lei Zhenyuan
9c07b203aa
[intel] change windows to remove windows-triton for intel xpu ( #3168 )
...
* change windows to remove windows-triton for intel xpu
* add changes for different platform
* Update pyproject.toml
* update mode windows
* Update pyproject.toml
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update pyproject.toml
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update pyproject.toml
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update pyproject.toml
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update pyproject.toml
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update pyproject.toml
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update pyproject.toml
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update pyproject.toml
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-11-23 22:03:54 -08:00
Etherll
3cbfa1573a
Add trust_remote_code parameter to tokenizer ( #3631 )
2025-11-23 21:12:40 -08:00
Daniel Han
8d7964f108
Version
2025-11-22 06:20:00 -08:00
andrewor14
7dfb239f22
Add 128x128 PerBlock FP8 + RL ( #3629 )
...
* Add 128x128 PerBlock FP8 + RL
**Summary:** Following https://github.com/unslothai/unsloth/pull/3440 ,
this PR extends torchao FP8 + RL support to also handle 128x128
PerBlock granularity (in addition to PerRow).
**Example usage:**
```
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "unsloth/Qwen3-8B-Base",
max_seq_length = 2048,
load_in_4bit = False,
fast_inference = True,
max_lora_rank = 32,
load_in_fp8 = "block", # or "row" or True
)
```
**Initial results:** TBD
**Note:**
- Requires https://github.com/pytorch/ao/pull/3370
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-11-21 20:09:27 -08:00
Mercury
874b5f0448
Fix missing code and support inputs_embeds only input. ( #3623 )
2025-11-20 07:56:52 -08:00
Daniel Han
9f19876a09
Update loader_utils.py
2025-11-20 04:04:19 -08:00
Daniel Han
25f25f9d30
Update loader_utils.py
2025-11-20 00:06:11 -08:00
Roland Tannous
b701d743ee
Fix/save torchao model loading logic ( #3621 )
...
* make loading gpt-oss-BF16 faster. Linked to unsloth-zoo PR #314
* fix model loading and clean merged model directory
* revert default quant
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* revert mapper.py
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-11-20 00:02:31 -08:00
Daniel Han
a458d59e91
Update __init__.py
2025-11-19 23:56:38 -08:00
andrewor14
89d8677c0c
Enable FP8 + RL training for bf16 models ( #3440 )
...
* Enable FP8 + RL training for bf16 models
**Summary:** Enable FP8 + RL training using TorchAO for 1.33x faster training and 42% less model memory usage:
- We quantize the frozen LoRA weights into fp8 and keep the LoRA adapters in bf16
- We leverage TorchAO's `Float8Tensor`, which calls into fbgemm's fp8 x fp8 rowwise matmul kernel
- For now, we need to do an offline quantization first, because vllm doesn't support on-the-fly quantization for torchao yet (this is in progress: https://github.com/vllm-project/vllm/pull/26327 )
**Example usage:**
```
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "unsloth/Qwen3-8B-Base",
max_seq_length = 2048,
load_in_4bit = False,
fast_inference = True,
max_lora_rank = 32,
load_in_fp8 = True, # set this to True
)
\# the rest is the same as before
model = FastLanguageModel.get_peft_model(...)
```
**Initial results:**
```
\# fp8
{'train_runtime': 1725.4337, 'train_samples_per_second': 0.232, 'train_steps_per_second': 0.058, 'train_loss': 0.00015715716748673002, 'epoch': 0.01}
\# bf16
{'train_runtime': 2297.8145, 'train_samples_per_second': 0.174, 'train_steps_per_second': 0.044, 'train_loss': 0.00016081033063528594, 'epoch': 0.01}
```
<img width="1199" height="448" alt="Screenshot 2025-11-11 at 4 10 50 PM" src="https://github.com/user-attachments/assets/b6304afd-89e9-42b1-8064-775807e17b23 " />
Test script: https://gist.github.com/andrewor14/5b85119fae46845d07b608d420907423
**Requires:**
- https://github.com/pytorch/ao/pull/3158 (torchao nightly or 0.15.0+)
- https://github.com/unslothai/unsloth-zoo/pull/351
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update utils.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* _get_inference_mode_context_manager
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update utils.py
* Update utils.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2025-11-19 23:51:43 -08:00
DoubleMathew
ecacc17f7e
Remove grpo requirement bs=num_generations ( #3609 )
...
* Remove grpo requirement bs=num_generations
* Update rl.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-11-19 19:57:01 -08:00
DoubleMathew
0362092b1b
Add an int64 path for mlp kernels ( #3614 )
...
* Add an int64 path for mlp kernels
* move constant expressions to globals
* fix name
2025-11-19 19:45:10 -08:00
Dan Saunders
6e16ef3d4c
remove pre-commit workflow (covered by pre-commit app) ( #3618 )
2025-11-19 15:34:32 -08:00
mk0walsk
0cf5762060
Fix broken links and typo in README ( #3611 )
...
* README Link Fixes
* Update README.md
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-11-18 20:04:14 -08:00
pre-commit-ci[bot]
98d04047dd
[pre-commit.ci] pre-commit autoupdate ( #3606 )
...
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.14.4 → v0.14.5](https://github.com/astral-sh/ruff-pre-commit/compare/v0.14.4...v0.14.5 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-11-17 17:02:44 -08:00
Datta Nimmaturi
0f0b1fba15
Do not force set beta to 0 for DAPO ( #3604 )
2025-11-16 22:39:36 -08:00
DoubleMathew
3c963f811a
fix qwen3 vl gradient accumulation ( #3598 )
...
* fix qwen3 vl gradient accumulation
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update unsloth/models/_utils.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-11-15 02:26:34 -08:00
Daniel Han
22e7288271
Update pyproject.toml
2025-11-14 20:01:02 -08:00
Scott Roy
402c004e46
Extend TorchAOConfig to support mobile usecases ( #3587 )
...
* up
* up
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-11-14 03:08:21 -08:00
Yuxiao Cheng
64f40dbf12
Fix: prevent rope_embedding AssertionError by checking kv_seq_len before reuse ( #3578 )
...
* fix: add kv_seq_len boundary check before reusing RoPE embeddings
Prevented AssertionError in rope_embedding.forward when kv_seq_len exceeds
the cached rope size. Added condition to verify kv_seq_len <=
position_embeddings[0].shape[0] before reuse, ensuring dynamic extension
triggers correctly.
Fixes #3036 #3216
* fix falcon h1
---------
Co-authored-by: jarrycyx <dzdzzd@126.com>
2025-11-14 03:06:33 -08:00
Giuseppe Franco
fa5b05b70b
Support for out-of-source quantizers ( #3534 )
...
* Support for out-of-source quantizers
* Fix decorators and functions to be staticmethod
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-11-14 02:52:24 -08:00
DoubleMathew
9e932030ff
Patch in tiled mlp ( #3584 )
...
* Patch in tiled mlp
* Update unsloth/models/llama.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-11-13 21:26:49 -08:00
DoubleMathew
0dbff7bade
Resize rope embeddings for long sequence training ( #3586 )
2025-11-11 18:11:31 -08:00
pre-commit-ci[bot]
ff16d280f9
[pre-commit.ci] pre-commit autoupdate ( #3576 )
...
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.14.0 → v0.14.4](https://github.com/astral-sh/ruff-pre-commit/compare/v0.14.0...v0.14.4 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-11-11 18:10:49 -08:00
Daniel Han
3ef5b4d3c3
Update _utils.py
2025-11-10 04:49:27 -08:00
Dan Saunders
63821d4f7b
pre-commit CI config ( #3565 )
2025-11-07 14:44:18 -08:00
DoubleMathew
5d2f119bea
add trust_remote_code kwarg ( #3564 )
2025-11-07 14:16:35 -08:00
Daniel Han
86208411e4
Formatting & bug fixes ( #3563 )
...
* Update rl.py
* Fix CE Loss
* Versioning
* Update loader.py
* Update loader.py
* extract_model_type_from_config
* Model types
* Update loader.py
* get_transformers_model_type
* Update loader.py
* Update loader.py
* Update loader.py
* Update rl.py
* Update pyproject.toml
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Versioning
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update vision.py
* Update vision.py
* Fix DataParallel
* Update _utils.py
* Update rl.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update mapper.py
* Versioning
* Update loader.py
* Update loader.py
* Update rl.py
* Versioning
* Update _utils.py
* Fix auto_mapping
* Update loader.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update loader.py
* Message
* Update vision.py
* Update loader.py
* Update vision.py
* cache_implementation
* Update vision.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Save max_seq_length
* Update _utils.py
* Update rl.py
* Update vision.py
* Update llama.py
* Mistral3 vllm (#3349 )
* [WIP] use vLLM for vision language models
* Update README.md
Editing icon sizes
* Update README.md
Updating icon sizes
* Update README.md (#2885 )
* MoE kernels AGPLv3
* versioning
* Many bug fixes (#2908 )
* add deepseek v3
* add deepseek r1 base
* add deepseek r1 zero
* add deepseek distill llama
* add deepseek distill models
* remove redundant code when constructing model names
* add mistral small to registry
* rename model registration methods
* rename deepseek registration methods
* refactor naming for mistral and phi
* add global register models
* refactor model registration tests for new registry apis
* add model search method
* remove deprecated registration api
* add quant type test
* add registry readme
* make llama registration more specific
* clear registry when executing individual model registration file
* more registry readme updates
* Update _auto_install.py
* Llama4
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Synthetic data
* Update mapper.py
* Xet and Synthetic
* Update synthetic.py
* Update loader.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update pyproject.toml
* Delete .gitignore
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update _utils.py
* Update pyproject.toml
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update chat_templates.py
* Seasame force float16 / float32
* Fix Seasame
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* is_multimodal
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* UNSLOTH_DISABLE_STATIC_GENERATION
* Update vision.py
* Auto vision detection
* Sesame
* Whisper
* Update loader.py
* Update loader.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update _utils.py
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* logging
* Update pyproject.toml
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* logits / temperature
* Update rl_replacements.py
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Debugging only
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Generic efficient GRPO
* Update rl_replacements.py
* Update rl_replacements.py
* Remove debugging
* Update rl_replacements.py
* Update rl_replacements.py
* Update vision.py
* Update llama.py
* Update rl_replacements.py
* versioning
* Update _utils.py
* Update vision.py
* Update mapper.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update loader.py
* Update _utils.py
* Update vision.py
* gradient checkpointing
* Gemma 3N fixes
* Update loader.py
* Versioning
* Gemma 3N fixes
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Fix setup.py
* setup.py
* Prints
* Update setup.py
* Update setup.py
* Update setup.py
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update vision.py
* Update vision.py
* Update pyproject.toml
* Update vision.py
* Update _utils.py
* Update __init__.py
* Update __init__.py
---------
Co-authored-by: jeromeku <jerome.ku@gmail.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
* silienty skip falcon h1 import is transformers_version < 4.53.0 (#2912 )
* Dynamically adjust get_per_token_logps function and patch as well (#2911 )
* add intel gpu with vllm support (#2903 )
* [bugs] fix for casual mask (#2868 )
* fix for casual mask
* use un_casual in sdpa
* add missing mask
* fix for type
* Explicitly check if xformers exists for attention (#2889 )
* Update __init__.py
* Update llama.py
* if mlp doesn't exist in layer module check for feed_forward name for falcon h1 (#2913 )
* Move inputs to right devices. (#2919 )
* Move tensors to right devices
* fix multi gpu for non mistral models
* multi GPU RoPE for gemma2
* Finish up multi GPU inference
* Make multiGPU rope a list
* Remove unnecessary transfer to CPU
* Remove unnecessary move to CPU
* Donot move inputs to device yet
will be handled separately in another PR
* Move inputs to appropriate decoder device
* Make device count global variable
* Cleanup RoPE device code
* Fixup num_gpu to device count
* Cleanup device counts
* Use device index for RoPE get_cache
* Donot typecast
* Use tuple instead of list for tensors. Use device index directly
* fixup move to device logic
* WIP VLM vLLM
* Make vLLM patch a function
* Add save and load lora functions
* Make fast_inference setup depend on the flag
* Improve fast inference patching mechanism
* Make vision setting depend on checks in fastbasemodel
* Check LoRA and vLLM intercompatibility for vision models
* Comment pointing to vLLM LoRA check
* Improve lora validation on vLLM
* Error out on no vLLM and increase max lora rank
* Bug fixes (#3017 )
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update pyproject.toml
* Delete .gitignore
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update _utils.py
* Update pyproject.toml
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update chat_templates.py
* Seasame force float16 / float32
* Fix Seasame
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* is_multimodal
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* UNSLOTH_DISABLE_STATIC_GENERATION
* Update vision.py
* Auto vision detection
* Sesame
* Whisper
* Update loader.py
* Update loader.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update _utils.py
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* logging
* Update pyproject.toml
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* logits / temperature
* Update rl_replacements.py
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Debugging only
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Generic efficient GRPO
* Update rl_replacements.py
* Update rl_replacements.py
* Remove debugging
* Update rl_replacements.py
* Update rl_replacements.py
* Update vision.py
* Update llama.py
* Update rl_replacements.py
* versioning
* Update _utils.py
* Update vision.py
* Update mapper.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update loader.py
* Update _utils.py
* Update vision.py
* gradient checkpointing
* Gemma 3N fixes
* Update loader.py
* Versioning
* Gemma 3N fixes
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Fix setup.py
* setup.py
* Prints
* Update setup.py
* Update setup.py
* Update setup.py
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update vision.py
* Update vision.py
* Update pyproject.toml
* Update vision.py
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Small fixes
* Update vision.py
* Update vision.py
* versioning
* Update __init__.py
* Update llama.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update vision.py
* Update vision.py
* compiler stance
* Update _utils.py
* Update pyproject.toml
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Revert "Revert "Add Qwen2.5-VL-32B-Instruct mapping to fix quantized model me…" (#2990 )
This reverts commit 6631007493#3011 )
* [intel] add for intel path for llama.py (#3012 )
* fix for intel path
* remove unuse code
* Update unsloth/models/llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update llama.py
* Fix Gemma 2 (#3024 )
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update pyproject.toml
* Delete .gitignore
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update _utils.py
* Update pyproject.toml
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update chat_templates.py
* Seasame force float16 / float32
* Fix Seasame
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* is_multimodal
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* UNSLOTH_DISABLE_STATIC_GENERATION
* Update vision.py
* Auto vision detection
* Sesame
* Whisper
* Update loader.py
* Update loader.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update _utils.py
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* logging
* Update pyproject.toml
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* logits / temperature
* Update rl_replacements.py
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Debugging only
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Generic efficient GRPO
* Update rl_replacements.py
* Update rl_replacements.py
* Remove debugging
* Update rl_replacements.py
* Update rl_replacements.py
* Update vision.py
* Update llama.py
* Update rl_replacements.py
* versioning
* Update _utils.py
* Update vision.py
* Update mapper.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update loader.py
* Update _utils.py
* Update vision.py
* gradient checkpointing
* Gemma 3N fixes
* Update loader.py
* Versioning
* Gemma 3N fixes
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Fix setup.py
* setup.py
* Prints
* Update setup.py
* Update setup.py
* Update setup.py
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update vision.py
* Update vision.py
* Update pyproject.toml
* Update vision.py
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Small fixes
* Update vision.py
* Update vision.py
* versioning
* Update __init__.py
* Update llama.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update vision.py
* Update vision.py
* compiler stance
* Update _utils.py
* Update pyproject.toml
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Revert "Revert "Add Qwen2.5-VL-32B-Instruct mapping to fix quantized model me…" (#2990 )
This reverts commit 6631007493#3026 )
* Fix torch compile issues (#3028 )
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update pyproject.toml
* Delete .gitignore
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update _utils.py
* Update pyproject.toml
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update chat_templates.py
* Seasame force float16 / float32
* Fix Seasame
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* is_multimodal
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* UNSLOTH_DISABLE_STATIC_GENERATION
* Update vision.py
* Auto vision detection
* Sesame
* Whisper
* Update loader.py
* Update loader.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update _utils.py
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* logging
* Update pyproject.toml
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* logits / temperature
* Update rl_replacements.py
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Debugging only
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Generic efficient GRPO
* Update rl_replacements.py
* Update rl_replacements.py
* Remove debugging
* Update rl_replacements.py
* Update rl_replacements.py
* Update vision.py
* Update llama.py
* Update rl_replacements.py
* versioning
* Update _utils.py
* Update vision.py
* Update mapper.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update loader.py
* Update _utils.py
* Update vision.py
* gradient checkpointing
* Gemma 3N fixes
* Update loader.py
* Versioning
* Gemma 3N fixes
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Fix setup.py
* setup.py
* Prints
* Update setup.py
* Update setup.py
* Update setup.py
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update vision.py
* Update vision.py
* Update pyproject.toml
* Update vision.py
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Small fixes
* Update vision.py
* Update vision.py
* versioning
* Update __init__.py
* Update llama.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update vision.py
* Update vision.py
* compiler stance
* Update _utils.py
* Update pyproject.toml
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Revert "Revert "Add Qwen2.5-VL-32B-Instruct mapping to fix quantized model me…" (#2990 )
This reverts commit 6631007493#3186 )
* Fix mamba
* Update loader.py
* Update vision.py
* Update loader.py
* Filter vLLM standby logs (#3131 )
* filter vLLM standby logs
* safeguard standby logger patch
* Update unsloth/models/_utils.py
* Update unsloth/models/_utils.py
* Update unsloth/models/_utils.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update loader.py
* Add scaler
* Update llama.py
* Update _utils.py
* Versioning
* GPT OSS fix
* GPT OSS fix
* Update loader.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update llama.py
* Update llama.py
* Update llama.py
* Versioning
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Upcast norms
* Update loader.py
* Update vision.py
* Upcast layernorms
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update save.py
* Update rl.py
* Update pyproject.toml
* Update rl.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update __init__.py
* Torch 2.8
* Update rl_replacements.py
---------
Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com>
* Update _auto_install.py
* Update pyproject.toml
* Update rl.py
* Protobuf issue
* Update pyproject.toml
* Fix extras transformers typo in pyproject.toml
* Update _utils.py
* Bug fixes (#3195 )
* Fix mamba
* Update loader.py
* Update vision.py
* Update loader.py
* Filter vLLM standby logs (#3131 )
* filter vLLM standby logs
* safeguard standby logger patch
* Update unsloth/models/_utils.py
* Update unsloth/models/_utils.py
* Update unsloth/models/_utils.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update loader.py
* Add scaler
* Update llama.py
* Update _utils.py
* Versioning
* GPT OSS fix
* GPT OSS fix
* Update loader.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update llama.py
* Update llama.py
* Update llama.py
* Versioning
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Upcast norms
* Update loader.py
* Update vision.py
* Upcast layernorms
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update save.py
* Update rl.py
* Update pyproject.toml
* Update rl.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update __init__.py
* Torch 2.8
* Update rl_replacements.py
* Update loader.py
* UNSLOTH_ENABLE_CCE
* Fix
* Update loader.py
* Update loader.py
* Update __init__.py
* Update __init__.py
* Update __init__.py
* Update __init__.py
* Import fixes
* Update loader.py
* Fix aimv2 issue
* Update loader.py
* Update import_fixes.py
* Update import_fixes.py
* Update loader.py
* Update loader.py
* Update loader.py
* Upgrade
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
---------
Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com>
* adallow float32 dtype in FastLanguageModel (#3204 )
* Update loader.py
* Update vision.py
* Suppress message and use unsloth sampling params
* Use trl sampling params for now
* Improve error message
* fixup quantized fast inference model name
* Add mistral 3 support
---------
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: jeromeku <jerome.ku@gmail.com>
Co-authored-by: DoubleMathew <mmathew23@gmail.com>
Co-authored-by: Lei Zhenyuan <zhenyuan.lei@intel.com>
Co-authored-by: parth2510 <parthguptapg7326@gmail.com>
* Set padding to 0
* Fix patch
* fixup patch (#3359 )
Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com>
* Update vision.py
* Versioning
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* MXFP4 dequant
* Update loader.py
* Update vision.py
* load_in_16bit
* Update vision.py
* Update vision.py
* Update vision.py
* Update rl.py
* Update vision.py
* offload_embedding
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update rl_replacements.py
* Update loader.py
* Fix padding issue
* Update pyproject.toml
* Update _utils.py
* Update pyproject.toml
* Update _utils.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* New models
* Update llama.py
* Versioning
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update llama.py
* Fix AMD
* Update _utils.py
* Update llama.py
* Update vision.py
* DEVICE_TYPE_TORCH
* Update __init__.py
* Update __init__.py
* Update _utils.py
* Move DEVICE_TYPE
* Update rl_replacements.py
* Update loader.py
* AMD install script
* Move AMD
* Update _amd_install.sh
* Update pyproject.toml
* Update pyproject.toml
* Delete _amd_install.sh
* Update device_type.py
* Update loader.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update tokenizer_utils.py
* Versioning
* Update pyproject.toml
* Update loader.py
* Update _utils.py
* Update pyproject.toml
* Update pyproject.toml
* Update _utils.py
* Update pyproject.toml
* Update _utils.py
* Update _utils.py
* Update loader.py
* Update _utils.py
* Update _utils.py
* local_files_only
* Cut Cross Entropy
* Update llama.py
* Update vision.py
* Update vision.py
* Update vision.py
* Qwen 3 VL vLLM (#3489 )
* Update __init__.py
* patch_torchao
* torchao_logger
* Update rl_replacements.py
* Fix
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Versioning
* fbgemm fp8 block quant support (>=1.4.0) (#3531 )
* fbgemm fp8 block quant support (>=1.4.0)
* Verify for fp8 support before proceeding
* Use unsloth zoo's Version and improve comments
* spacessss
* Update vision.py
* Update vision.py
* Update rl.py
* vllm_sampling_params
* Update rl.py
* Update rl.py
* Update rl.py
* Add `ruff` pre-commit hook and apply it (#3424 )
* Add Ruff pre-commit config and workflow
* Add kwarg spacing enforcement helper
* Apply Ruff formatting
* Update fp8.py
* Revert ruff on some files
* Update
* force-exclude = true
* Datasets issue
* Ruff
* Remove mapper
* Update mapper.py
* Update pyproject.toml
---------
Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: jeromeku <jerome.ku@gmail.com>
Co-authored-by: DoubleMathew <mmathew23@gmail.com>
Co-authored-by: Lei Zhenyuan <zhenyuan.lei@intel.com>
Co-authored-by: parth2510 <parthguptapg7326@gmail.com>
Co-authored-by: Dan Saunders <danjsaund@gmail.com>
2025-11-07 06:00:22 -08:00
mk0walsk
07f0dfe0dc
Fix typos in comment ( #3557 )
2025-11-05 19:29:36 -08:00
Michael Han
cc8c31fb38
Update README.md
2025-11-04 22:00:06 -08:00
pluesclues
d0183a3cc6
Detach logits before returning from function ( #3554 )
2025-11-04 07:29:27 -08:00
Datta Nimmaturi
b71a3b355b
Sleep trl patch ( #3517 )
...
* Patch sleep mode properly for trl
* empty cache after sleep/wakeup
* no extra wakeups
* Do not redo wakeups
* cleanup
* post trl 0.23 sleep patch
2025-11-03 23:00:54 -08:00
Daniel Han
e10f3e940d
Bug fixes ( #3546 )
...
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Bug fix
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* torch_dtype
* Update rl.py
* Fix CE Loss
* Versioning
* Update loader.py
* Update loader.py
* extract_model_type_from_config
* Model types
* Update loader.py
* get_transformers_model_type
* Update loader.py
* Update loader.py
* Update loader.py
* Update rl.py
* Update pyproject.toml
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Versioning
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update vision.py
* Update vision.py
* Fix DataParallel
* Update _utils.py
* Update rl.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update mapper.py
* Versioning
* Update loader.py
* Update loader.py
* Update rl.py
* Versioning
* Update _utils.py
* Fix auto_mapping
* Update loader.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update loader.py
* Message
* Update vision.py
* Update loader.py
* Update vision.py
* cache_implementation
* Update vision.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Save max_seq_length
* Update _utils.py
* Update rl.py
* Update vision.py
* Update llama.py
* Mistral3 vllm (#3349 )
* [WIP] use vLLM for vision language models
* Update README.md
Editing icon sizes
* Update README.md
Updating icon sizes
* Update README.md (#2885 )
* MoE kernels AGPLv3
* versioning
* Many bug fixes (#2908 )
* add deepseek v3
* add deepseek r1 base
* add deepseek r1 zero
* add deepseek distill llama
* add deepseek distill models
* remove redundant code when constructing model names
* add mistral small to registry
* rename model registration methods
* rename deepseek registration methods
* refactor naming for mistral and phi
* add global register models
* refactor model registration tests for new registry apis
* add model search method
* remove deprecated registration api
* add quant type test
* add registry readme
* make llama registration more specific
* clear registry when executing individual model registration file
* more registry readme updates
* Update _auto_install.py
* Llama4
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Synthetic data
* Update mapper.py
* Xet and Synthetic
* Update synthetic.py
* Update loader.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update pyproject.toml
* Delete .gitignore
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update _utils.py
* Update pyproject.toml
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update chat_templates.py
* Seasame force float16 / float32
* Fix Seasame
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* is_multimodal
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* UNSLOTH_DISABLE_STATIC_GENERATION
* Update vision.py
* Auto vision detection
* Sesame
* Whisper
* Update loader.py
* Update loader.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update _utils.py
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* logging
* Update pyproject.toml
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* logits / temperature
* Update rl_replacements.py
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Debugging only
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Generic efficient GRPO
* Update rl_replacements.py
* Update rl_replacements.py
* Remove debugging
* Update rl_replacements.py
* Update rl_replacements.py
* Update vision.py
* Update llama.py
* Update rl_replacements.py
* versioning
* Update _utils.py
* Update vision.py
* Update mapper.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update loader.py
* Update _utils.py
* Update vision.py
* gradient checkpointing
* Gemma 3N fixes
* Update loader.py
* Versioning
* Gemma 3N fixes
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Fix setup.py
* setup.py
* Prints
* Update setup.py
* Update setup.py
* Update setup.py
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update vision.py
* Update vision.py
* Update pyproject.toml
* Update vision.py
* Update _utils.py
* Update __init__.py
* Update __init__.py
---------
Co-authored-by: jeromeku <jerome.ku@gmail.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
* silienty skip falcon h1 import is transformers_version < 4.53.0 (#2912 )
* Dynamically adjust get_per_token_logps function and patch as well (#2911 )
* add intel gpu with vllm support (#2903 )
* [bugs] fix for casual mask (#2868 )
* fix for casual mask
* use un_casual in sdpa
* add missing mask
* fix for type
* Explicitly check if xformers exists for attention (#2889 )
* Update __init__.py
* Update llama.py
* if mlp doesn't exist in layer module check for feed_forward name for falcon h1 (#2913 )
* Move inputs to right devices. (#2919 )
* Move tensors to right devices
* fix multi gpu for non mistral models
* multi GPU RoPE for gemma2
* Finish up multi GPU inference
* Make multiGPU rope a list
* Remove unnecessary transfer to CPU
* Remove unnecessary move to CPU
* Donot move inputs to device yet
will be handled separately in another PR
* Move inputs to appropriate decoder device
* Make device count global variable
* Cleanup RoPE device code
* Fixup num_gpu to device count
* Cleanup device counts
* Use device index for RoPE get_cache
* Donot typecast
* Use tuple instead of list for tensors. Use device index directly
* fixup move to device logic
* WIP VLM vLLM
* Make vLLM patch a function
* Add save and load lora functions
* Make fast_inference setup depend on the flag
* Improve fast inference patching mechanism
* Make vision setting depend on checks in fastbasemodel
* Check LoRA and vLLM intercompatibility for vision models
* Comment pointing to vLLM LoRA check
* Improve lora validation on vLLM
* Error out on no vLLM and increase max lora rank
* Bug fixes (#3017 )
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update pyproject.toml
* Delete .gitignore
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update _utils.py
* Update pyproject.toml
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update chat_templates.py
* Seasame force float16 / float32
* Fix Seasame
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* is_multimodal
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* UNSLOTH_DISABLE_STATIC_GENERATION
* Update vision.py
* Auto vision detection
* Sesame
* Whisper
* Update loader.py
* Update loader.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update _utils.py
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* logging
* Update pyproject.toml
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* logits / temperature
* Update rl_replacements.py
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Debugging only
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Generic efficient GRPO
* Update rl_replacements.py
* Update rl_replacements.py
* Remove debugging
* Update rl_replacements.py
* Update rl_replacements.py
* Update vision.py
* Update llama.py
* Update rl_replacements.py
* versioning
* Update _utils.py
* Update vision.py
* Update mapper.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update loader.py
* Update _utils.py
* Update vision.py
* gradient checkpointing
* Gemma 3N fixes
* Update loader.py
* Versioning
* Gemma 3N fixes
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Fix setup.py
* setup.py
* Prints
* Update setup.py
* Update setup.py
* Update setup.py
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update vision.py
* Update vision.py
* Update pyproject.toml
* Update vision.py
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Small fixes
* Update vision.py
* Update vision.py
* versioning
* Update __init__.py
* Update llama.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update vision.py
* Update vision.py
* compiler stance
* Update _utils.py
* Update pyproject.toml
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Revert "Revert "Add Qwen2.5-VL-32B-Instruct mapping to fix quantized model me…" (#2990 )
This reverts commit 6631007493#3011 )
* [intel] add for intel path for llama.py (#3012 )
* fix for intel path
* remove unuse code
* Update unsloth/models/llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update llama.py
* Fix Gemma 2 (#3024 )
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update pyproject.toml
* Delete .gitignore
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update _utils.py
* Update pyproject.toml
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update chat_templates.py
* Seasame force float16 / float32
* Fix Seasame
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* is_multimodal
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* UNSLOTH_DISABLE_STATIC_GENERATION
* Update vision.py
* Auto vision detection
* Sesame
* Whisper
* Update loader.py
* Update loader.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update _utils.py
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* logging
* Update pyproject.toml
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* logits / temperature
* Update rl_replacements.py
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Debugging only
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Generic efficient GRPO
* Update rl_replacements.py
* Update rl_replacements.py
* Remove debugging
* Update rl_replacements.py
* Update rl_replacements.py
* Update vision.py
* Update llama.py
* Update rl_replacements.py
* versioning
* Update _utils.py
* Update vision.py
* Update mapper.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update loader.py
* Update _utils.py
* Update vision.py
* gradient checkpointing
* Gemma 3N fixes
* Update loader.py
* Versioning
* Gemma 3N fixes
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Fix setup.py
* setup.py
* Prints
* Update setup.py
* Update setup.py
* Update setup.py
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update vision.py
* Update vision.py
* Update pyproject.toml
* Update vision.py
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Small fixes
* Update vision.py
* Update vision.py
* versioning
* Update __init__.py
* Update llama.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update vision.py
* Update vision.py
* compiler stance
* Update _utils.py
* Update pyproject.toml
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Revert "Revert "Add Qwen2.5-VL-32B-Instruct mapping to fix quantized model me…" (#2990 )
This reverts commit 6631007493#3026 )
* Fix torch compile issues (#3028 )
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update pyproject.toml
* Delete .gitignore
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update _utils.py
* Update pyproject.toml
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update chat_templates.py
* Seasame force float16 / float32
* Fix Seasame
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* is_multimodal
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* UNSLOTH_DISABLE_STATIC_GENERATION
* Update vision.py
* Auto vision detection
* Sesame
* Whisper
* Update loader.py
* Update loader.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update _utils.py
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* logging
* Update pyproject.toml
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* logits / temperature
* Update rl_replacements.py
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Debugging only
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Generic efficient GRPO
* Update rl_replacements.py
* Update rl_replacements.py
* Remove debugging
* Update rl_replacements.py
* Update rl_replacements.py
* Update vision.py
* Update llama.py
* Update rl_replacements.py
* versioning
* Update _utils.py
* Update vision.py
* Update mapper.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update loader.py
* Update _utils.py
* Update vision.py
* gradient checkpointing
* Gemma 3N fixes
* Update loader.py
* Versioning
* Gemma 3N fixes
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Fix setup.py
* setup.py
* Prints
* Update setup.py
* Update setup.py
* Update setup.py
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update vision.py
* Update vision.py
* Update pyproject.toml
* Update vision.py
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Small fixes
* Update vision.py
* Update vision.py
* versioning
* Update __init__.py
* Update llama.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update vision.py
* Update vision.py
* compiler stance
* Update _utils.py
* Update pyproject.toml
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Revert "Revert "Add Qwen2.5-VL-32B-Instruct mapping to fix quantized model me…" (#2990 )
This reverts commit 6631007493#3186 )
* Fix mamba
* Update loader.py
* Update vision.py
* Update loader.py
* Filter vLLM standby logs (#3131 )
* filter vLLM standby logs
* safeguard standby logger patch
* Update unsloth/models/_utils.py
* Update unsloth/models/_utils.py
* Update unsloth/models/_utils.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update loader.py
* Add scaler
* Update llama.py
* Update _utils.py
* Versioning
* GPT OSS fix
* GPT OSS fix
* Update loader.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update llama.py
* Update llama.py
* Update llama.py
* Versioning
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Upcast norms
* Update loader.py
* Update vision.py
* Upcast layernorms
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update save.py
* Update rl.py
* Update pyproject.toml
* Update rl.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update __init__.py
* Torch 2.8
* Update rl_replacements.py
---------
Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com>
* Update _auto_install.py
* Update pyproject.toml
* Update rl.py
* Protobuf issue
* Update pyproject.toml
* Fix extras transformers typo in pyproject.toml
* Update _utils.py
* Bug fixes (#3195 )
* Fix mamba
* Update loader.py
* Update vision.py
* Update loader.py
* Filter vLLM standby logs (#3131 )
* filter vLLM standby logs
* safeguard standby logger patch
* Update unsloth/models/_utils.py
* Update unsloth/models/_utils.py
* Update unsloth/models/_utils.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update loader.py
* Add scaler
* Update llama.py
* Update _utils.py
* Versioning
* GPT OSS fix
* GPT OSS fix
* Update loader.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update llama.py
* Update llama.py
* Update llama.py
* Versioning
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Upcast norms
* Update loader.py
* Update vision.py
* Upcast layernorms
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update save.py
* Update rl.py
* Update pyproject.toml
* Update rl.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update __init__.py
* Torch 2.8
* Update rl_replacements.py
* Update loader.py
* UNSLOTH_ENABLE_CCE
* Fix
* Update loader.py
* Update loader.py
* Update __init__.py
* Update __init__.py
* Update __init__.py
* Update __init__.py
* Import fixes
* Update loader.py
* Fix aimv2 issue
* Update loader.py
* Update import_fixes.py
* Update import_fixes.py
* Update loader.py
* Update loader.py
* Update loader.py
* Upgrade
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
---------
Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com>
* adallow float32 dtype in FastLanguageModel (#3204 )
* Update loader.py
* Update vision.py
* Suppress message and use unsloth sampling params
* Use trl sampling params for now
* Improve error message
* fixup quantized fast inference model name
* Add mistral 3 support
---------
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: jeromeku <jerome.ku@gmail.com>
Co-authored-by: DoubleMathew <mmathew23@gmail.com>
Co-authored-by: Lei Zhenyuan <zhenyuan.lei@intel.com>
Co-authored-by: parth2510 <parthguptapg7326@gmail.com>
* Set padding to 0
* Fix patch
* fixup patch (#3359 )
Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com>
* Update vision.py
* Versioning
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* MXFP4 dequant
* Update loader.py
* Update vision.py
* load_in_16bit
* Update vision.py
* Update vision.py
* Update vision.py
* Update rl.py
* Update vision.py
* offload_embedding
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update rl_replacements.py
* Update loader.py
* Fix padding issue
* Update pyproject.toml
* Update _utils.py
* Update pyproject.toml
* Update _utils.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* New models
* Update llama.py
* Versioning
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update llama.py
* Fix AMD
* Update _utils.py
* Update llama.py
* Update vision.py
* DEVICE_TYPE_TORCH
* Update __init__.py
* Update __init__.py
* Update _utils.py
* Move DEVICE_TYPE
* Update rl_replacements.py
* Update loader.py
* AMD install script
* Move AMD
* Update _amd_install.sh
* Update pyproject.toml
* Update pyproject.toml
* Delete _amd_install.sh
* Update device_type.py
* Update loader.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update tokenizer_utils.py
* Versioning
* Update pyproject.toml
* Update loader.py
* Update _utils.py
* Update pyproject.toml
* Update pyproject.toml
* Update _utils.py
* Update pyproject.toml
* Update _utils.py
* Update _utils.py
* Update loader.py
* Update _utils.py
* Update _utils.py
* local_files_only
* Cut Cross Entropy
* Update llama.py
* Update vision.py
* Update vision.py
* Update vision.py
* Qwen 3 VL vLLM (#3489 )
* Update __init__.py
* patch_torchao
* torchao_logger
* Update rl_replacements.py
* Fix
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Versioning
---------
Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: jeromeku <jerome.ku@gmail.com>
Co-authored-by: DoubleMathew <mmathew23@gmail.com>
Co-authored-by: Lei Zhenyuan <zhenyuan.lei@intel.com>
Co-authored-by: parth2510 <parthguptapg7326@gmail.com>
2025-11-03 06:47:26 -08:00
pluesclues
35f903b42d
Handle TRL version compatibility in rl_replacements.py ( #3540 )
2025-11-01 05:17:27 -07:00
Daniel Han
ad5fa96611
Update mapper.py
2025-10-30 06:56:22 -07:00
Daniel Han
e7b7bb95df
Update pyproject.toml
2025-10-30 06:48:14 -07:00
Daniel Han
c673919449
Nightly ( #3532 )
...
* Update loader.py
* Update vision.py
* Update vision.py
* custom_datatype
* recheck
* Float16
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Bug fix
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* torch_dtype
* Update rl.py
* Fix CE Loss
* Versioning
* Update loader.py
* Update loader.py
* extract_model_type_from_config
* Model types
* Update loader.py
* get_transformers_model_type
* Update loader.py
* Update loader.py
* Update loader.py
* Update rl.py
* Update pyproject.toml
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Versioning
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update vision.py
* Update vision.py
* Fix DataParallel
* Update _utils.py
* Update rl.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update mapper.py
* Versioning
* Update loader.py
* Update loader.py
* Update rl.py
* Versioning
* Update _utils.py
* Fix auto_mapping
* Update loader.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update loader.py
* Message
* Update vision.py
* Update loader.py
* Update vision.py
* cache_implementation
* Update vision.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Save max_seq_length
* Update _utils.py
* Update rl.py
* Update vision.py
* Update llama.py
* Mistral3 vllm (#3349 )
* [WIP] use vLLM for vision language models
* Update README.md
Editing icon sizes
* Update README.md
Updating icon sizes
* Update README.md (#2885 )
* MoE kernels AGPLv3
* versioning
* Many bug fixes (#2908 )
* add deepseek v3
* add deepseek r1 base
* add deepseek r1 zero
* add deepseek distill llama
* add deepseek distill models
* remove redundant code when constructing model names
* add mistral small to registry
* rename model registration methods
* rename deepseek registration methods
* refactor naming for mistral and phi
* add global register models
* refactor model registration tests for new registry apis
* add model search method
* remove deprecated registration api
* add quant type test
* add registry readme
* make llama registration more specific
* clear registry when executing individual model registration file
* more registry readme updates
* Update _auto_install.py
* Llama4
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Synthetic data
* Update mapper.py
* Xet and Synthetic
* Update synthetic.py
* Update loader.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update pyproject.toml
* Delete .gitignore
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update _utils.py
* Update pyproject.toml
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update chat_templates.py
* Seasame force float16 / float32
* Fix Seasame
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* is_multimodal
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* UNSLOTH_DISABLE_STATIC_GENERATION
* Update vision.py
* Auto vision detection
* Sesame
* Whisper
* Update loader.py
* Update loader.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update _utils.py
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* logging
* Update pyproject.toml
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* logits / temperature
* Update rl_replacements.py
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Debugging only
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Generic efficient GRPO
* Update rl_replacements.py
* Update rl_replacements.py
* Remove debugging
* Update rl_replacements.py
* Update rl_replacements.py
* Update vision.py
* Update llama.py
* Update rl_replacements.py
* versioning
* Update _utils.py
* Update vision.py
* Update mapper.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update loader.py
* Update _utils.py
* Update vision.py
* gradient checkpointing
* Gemma 3N fixes
* Update loader.py
* Versioning
* Gemma 3N fixes
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Fix setup.py
* setup.py
* Prints
* Update setup.py
* Update setup.py
* Update setup.py
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update vision.py
* Update vision.py
* Update pyproject.toml
* Update vision.py
* Update _utils.py
* Update __init__.py
* Update __init__.py
---------
Co-authored-by: jeromeku <jerome.ku@gmail.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
* silienty skip falcon h1 import is transformers_version < 4.53.0 (#2912 )
* Dynamically adjust get_per_token_logps function and patch as well (#2911 )
* add intel gpu with vllm support (#2903 )
* [bugs] fix for casual mask (#2868 )
* fix for casual mask
* use un_casual in sdpa
* add missing mask
* fix for type
* Explicitly check if xformers exists for attention (#2889 )
* Update __init__.py
* Update llama.py
* if mlp doesn't exist in layer module check for feed_forward name for falcon h1 (#2913 )
* Move inputs to right devices. (#2919 )
* Move tensors to right devices
* fix multi gpu for non mistral models
* multi GPU RoPE for gemma2
* Finish up multi GPU inference
* Make multiGPU rope a list
* Remove unnecessary transfer to CPU
* Remove unnecessary move to CPU
* Donot move inputs to device yet
will be handled separately in another PR
* Move inputs to appropriate decoder device
* Make device count global variable
* Cleanup RoPE device code
* Fixup num_gpu to device count
* Cleanup device counts
* Use device index for RoPE get_cache
* Donot typecast
* Use tuple instead of list for tensors. Use device index directly
* fixup move to device logic
* WIP VLM vLLM
* Make vLLM patch a function
* Add save and load lora functions
* Make fast_inference setup depend on the flag
* Improve fast inference patching mechanism
* Make vision setting depend on checks in fastbasemodel
* Check LoRA and vLLM intercompatibility for vision models
* Comment pointing to vLLM LoRA check
* Improve lora validation on vLLM
* Error out on no vLLM and increase max lora rank
* Bug fixes (#3017 )
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update pyproject.toml
* Delete .gitignore
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update _utils.py
* Update pyproject.toml
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update chat_templates.py
* Seasame force float16 / float32
* Fix Seasame
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* is_multimodal
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* UNSLOTH_DISABLE_STATIC_GENERATION
* Update vision.py
* Auto vision detection
* Sesame
* Whisper
* Update loader.py
* Update loader.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update _utils.py
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* logging
* Update pyproject.toml
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* logits / temperature
* Update rl_replacements.py
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Debugging only
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Generic efficient GRPO
* Update rl_replacements.py
* Update rl_replacements.py
* Remove debugging
* Update rl_replacements.py
* Update rl_replacements.py
* Update vision.py
* Update llama.py
* Update rl_replacements.py
* versioning
* Update _utils.py
* Update vision.py
* Update mapper.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update loader.py
* Update _utils.py
* Update vision.py
* gradient checkpointing
* Gemma 3N fixes
* Update loader.py
* Versioning
* Gemma 3N fixes
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Fix setup.py
* setup.py
* Prints
* Update setup.py
* Update setup.py
* Update setup.py
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update vision.py
* Update vision.py
* Update pyproject.toml
* Update vision.py
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Small fixes
* Update vision.py
* Update vision.py
* versioning
* Update __init__.py
* Update llama.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update vision.py
* Update vision.py
* compiler stance
* Update _utils.py
* Update pyproject.toml
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Revert "Revert "Add Qwen2.5-VL-32B-Instruct mapping to fix quantized model me…" (#2990 )
This reverts commit 6631007493#3011 )
* [intel] add for intel path for llama.py (#3012 )
* fix for intel path
* remove unuse code
* Update unsloth/models/llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update llama.py
* Fix Gemma 2 (#3024 )
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update pyproject.toml
* Delete .gitignore
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update _utils.py
* Update pyproject.toml
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update chat_templates.py
* Seasame force float16 / float32
* Fix Seasame
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* is_multimodal
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* UNSLOTH_DISABLE_STATIC_GENERATION
* Update vision.py
* Auto vision detection
* Sesame
* Whisper
* Update loader.py
* Update loader.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update _utils.py
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* logging
* Update pyproject.toml
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* logits / temperature
* Update rl_replacements.py
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Debugging only
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Generic efficient GRPO
* Update rl_replacements.py
* Update rl_replacements.py
* Remove debugging
* Update rl_replacements.py
* Update rl_replacements.py
* Update vision.py
* Update llama.py
* Update rl_replacements.py
* versioning
* Update _utils.py
* Update vision.py
* Update mapper.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update loader.py
* Update _utils.py
* Update vision.py
* gradient checkpointing
* Gemma 3N fixes
* Update loader.py
* Versioning
* Gemma 3N fixes
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Fix setup.py
* setup.py
* Prints
* Update setup.py
* Update setup.py
* Update setup.py
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update vision.py
* Update vision.py
* Update pyproject.toml
* Update vision.py
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Small fixes
* Update vision.py
* Update vision.py
* versioning
* Update __init__.py
* Update llama.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update vision.py
* Update vision.py
* compiler stance
* Update _utils.py
* Update pyproject.toml
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Revert "Revert "Add Qwen2.5-VL-32B-Instruct mapping to fix quantized model me…" (#2990 )
This reverts commit 6631007493#3026 )
* Fix torch compile issues (#3028 )
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update pyproject.toml
* Delete .gitignore
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update _utils.py
* Update pyproject.toml
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update synthetic.py
* Update chat_templates.py
* Seasame force float16 / float32
* Fix Seasame
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* is_multimodal
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update vision.py
* UNSLOTH_DISABLE_STATIC_GENERATION
* Update vision.py
* Auto vision detection
* Sesame
* Whisper
* Update loader.py
* Update loader.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update _utils.py
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* logging
* Update pyproject.toml
* Update rl.py
* versioning
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* logits / temperature
* Update rl_replacements.py
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Debugging only
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Generic efficient GRPO
* Update rl_replacements.py
* Update rl_replacements.py
* Remove debugging
* Update rl_replacements.py
* Update rl_replacements.py
* Update vision.py
* Update llama.py
* Update rl_replacements.py
* versioning
* Update _utils.py
* Update vision.py
* Update mapper.py
* Update loader.py
* Update mapper.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update loader.py
* Update _utils.py
* Update vision.py
* gradient checkpointing
* Gemma 3N fixes
* Update loader.py
* Versioning
* Gemma 3N fixes
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Fix setup.py
* setup.py
* Prints
* Update setup.py
* Update setup.py
* Update setup.py
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update pyproject.toml
* Update vision.py
* Update vision.py
* Update pyproject.toml
* Update vision.py
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Small fixes
* Update vision.py
* Update vision.py
* versioning
* Update __init__.py
* Update llama.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update vision.py
* Update vision.py
* compiler stance
* Update _utils.py
* Update pyproject.toml
* Update pyproject.toml
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Revert "Revert "Add Qwen2.5-VL-32B-Instruct mapping to fix quantized model me…" (#2990 )
This reverts commit 6631007493#3186 )
* Fix mamba
* Update loader.py
* Update vision.py
* Update loader.py
* Filter vLLM standby logs (#3131 )
* filter vLLM standby logs
* safeguard standby logger patch
* Update unsloth/models/_utils.py
* Update unsloth/models/_utils.py
* Update unsloth/models/_utils.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update loader.py
* Add scaler
* Update llama.py
* Update _utils.py
* Versioning
* GPT OSS fix
* GPT OSS fix
* Update loader.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update llama.py
* Update llama.py
* Update llama.py
* Versioning
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Upcast norms
* Update loader.py
* Update vision.py
* Upcast layernorms
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update save.py
* Update rl.py
* Update pyproject.toml
* Update rl.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update __init__.py
* Torch 2.8
* Update rl_replacements.py
---------
Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com>
* Update _auto_install.py
* Update pyproject.toml
* Update rl.py
* Protobuf issue
* Update pyproject.toml
* Fix extras transformers typo in pyproject.toml
* Update _utils.py
* Bug fixes (#3195 )
* Fix mamba
* Update loader.py
* Update vision.py
* Update loader.py
* Filter vLLM standby logs (#3131 )
* filter vLLM standby logs
* safeguard standby logger patch
* Update unsloth/models/_utils.py
* Update unsloth/models/_utils.py
* Update unsloth/models/_utils.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update loader.py
* Add scaler
* Update llama.py
* Update _utils.py
* Versioning
* GPT OSS fix
* GPT OSS fix
* Update loader.py
* Update vision.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update vision.py
* Update llama.py
* Update llama.py
* Update llama.py
* Versioning
* Update mapper.py
* Update vision.py
* Update vision.py
* Update vision.py
* Upcast norms
* Update loader.py
* Update vision.py
* Upcast layernorms
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update save.py
* Update rl.py
* Update pyproject.toml
* Update rl.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update __init__.py
* Torch 2.8
* Update rl_replacements.py
* Update loader.py
* UNSLOTH_ENABLE_CCE
* Fix
* Update loader.py
* Update loader.py
* Update __init__.py
* Update __init__.py
* Update __init__.py
* Update __init__.py
* Import fixes
* Update loader.py
* Fix aimv2 issue
* Update loader.py
* Update import_fixes.py
* Update import_fixes.py
* Update loader.py
* Update loader.py
* Update loader.py
* Upgrade
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
---------
Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com>
* adallow float32 dtype in FastLanguageModel (#3204 )
* Update loader.py
* Update vision.py
* Suppress message and use unsloth sampling params
* Use trl sampling params for now
* Improve error message
* fixup quantized fast inference model name
* Add mistral 3 support
---------
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: jeromeku <jerome.ku@gmail.com>
Co-authored-by: DoubleMathew <mmathew23@gmail.com>
Co-authored-by: Lei Zhenyuan <zhenyuan.lei@intel.com>
Co-authored-by: parth2510 <parthguptapg7326@gmail.com>
* Set padding to 0
* Fix patch
* fixup patch (#3359 )
Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com>
* Update vision.py
* Versioning
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* MXFP4 dequant
* Update loader.py
* Update vision.py
* load_in_16bit
* Update vision.py
* Update vision.py
* Update vision.py
* Update rl.py
* Update vision.py
* offload_embedding
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update rl_replacements.py
* Update loader.py
* Fix padding issue
* Update pyproject.toml
* Update _utils.py
* Update pyproject.toml
* Update _utils.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* New models
* Update llama.py
* Versioning
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update llama.py
* Fix AMD
* Update _utils.py
* Update llama.py
* Update vision.py
* DEVICE_TYPE_TORCH
* Update __init__.py
* Update __init__.py
* Update _utils.py
* Move DEVICE_TYPE
* Update rl_replacements.py
* Update loader.py
* AMD install script
* Move AMD
* Update _amd_install.sh
* Update pyproject.toml
* Update pyproject.toml
* Delete _amd_install.sh
* Update device_type.py
* Update loader.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update tokenizer_utils.py
* Versioning
* Update pyproject.toml
* Update loader.py
* Update _utils.py
* Update pyproject.toml
* Update pyproject.toml
* Update _utils.py
* Update pyproject.toml
* Update _utils.py
* Update _utils.py
* Update loader.py
* Update _utils.py
* Update _utils.py
* local_files_only
* Cut Cross Entropy
* Update llama.py
* Update vision.py
* Update vision.py
* Update vision.py
---------
Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: jeromeku <jerome.ku@gmail.com>
Co-authored-by: DoubleMathew <mmathew23@gmail.com>
Co-authored-by: Lei Zhenyuan <zhenyuan.lei@intel.com>
Co-authored-by: parth2510 <parthguptapg7326@gmail.com>
2025-10-30 06:45:57 -07:00
Daniel Han
85d73ce38c
Update vision.py
2025-10-30 06:30:43 -07:00
Daniel Han
4c8ba55ef1
Update vision.py
2025-10-30 06:28:05 -07:00
Daniel Han
a414ca8258
Update vision.py
2025-10-30 06:27:58 -07:00
Daniel Han
372d77fc66
Update vision.py
2025-10-30 06:27:31 -07:00
Daniel Han
06aad28f4c
Update vision.py
2025-10-30 06:23:28 -07:00
Daniel Han