* Implement GGUF upload method for SentenceTransformer
Added a method to convert and upload SentenceTransformer models to GGUF format, including handling of tokenizer, quantization methods, and repository management on Hugging Face Hub.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
On Windows and macOS (Python 3.8+), multiprocessing uses the spawn
start method. When datasets .map(num_proc=N) is called, it creates a
Pool(N) which re-imports __main__ in each worker, causing infinite
recursion and a RuntimeError during bootstrapping.
Guard the auto-computed dataset_num_proc in the generated Config
__init__ by checking multiprocessing.get_start_method() != 'fork'.
When the start method is not fork (spawn/forkserver), force
dataset_num_proc = None so datasets takes the single-process path.
Linux fork behavior is unchanged.
Also replace the fixed memory threshold logic with the simpler
adaptive approach: cap at 64, then min(num_proc, int(available_gb)),
with a safety floor of 1 when available memory is at or below 2GB.
Co-authored-by: Daniel Hanchen <danielhanchen@users.noreply.github.com>
* Disable torchcodec in transformers when FFmpeg is missing
When torchcodec is installed but FFmpeg libraries are unavailable,
transformers still thinks torchcodec is available (via find_spec check)
and tries to use it for audio loading, causing RuntimeError.
This adds disable_torchcodec_if_broken() which tests if torchcodec can
actually load its native libraries, and if not, patches transformers'
_torchcodec_available to False so it falls back to librosa instead.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The cuda.cutlass_epilogue_fusion_enabled and cuda.cutlass_tma_only
inductor config options were added in PyTorch 2.8.0. Using these
options on older PyTorch versions causes a RuntimeError during
GRPOTrainer initialization.
This fix adds a version check to only include these options when
running PyTorch 2.8.0 or later, allowing GRPO training to work on
older PyTorch versions (e.g., Colab environments with PyTorch 2.5-2.7).
Co-authored-by: Daniel Hanchen <danielhanchen@users.noreply.github.com>
When datasets library has torchcodec installed but FFmpeg libraries
are missing, torchcodec raises a RuntimeError during import. The
exception handler only caught ImportError and AttributeError, causing
the error to propagate and crash Unsloth imports in environments
like Colab where FFmpeg may not be installed.
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
* Improve MoE performance
* small changes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix imports
* disable autotune
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* LoRA for MoE
* Make autotune default
* make dy contiguous
* use non lora model as base for RL
* Revert "use non lora model as base for RL"
This reverts commit bc8f15629d060593b2eaf436f158ff5ac9df0d5d.
* fixup derp
* non TMA [T4]
* Revert "non TMA [T4]"
This reverts commit 35304566690e7c9ab9632899920c85bff322409a.
* Fixes for VL MoE and v5 transformers
* [transformers] [v5] remove unused hybridcache (#3910)
* remote unused hybridcache
* cleanup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* No double compile for qwen3moe
* Fix top_k on trl GRPO
* Recognise GLM as MoE
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix missing RotaryEmbeddingConfigMixin
* Licensing for autotuning cache
* Cleanup
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
_patch_trl_rl_trainers enumerates all trainer modules from dir(trl.trainer)
and attempts to import each one. Modules like alignprop_trainer fail because
they depend on optional packages (diffusers) that may not be installed. The
failure is harmless but the print() call produces noise on every import.
Change print() to logger.info() so these messages only appear when
UNSLOTH_ENABLE_LOGGING=1.
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
GPT-OSS models use eager attention during inference because flex
attention returns incorrect results (likely due to left padding).
However, when _attn_implementation is set to "flex_attention",
transformers creates BlockMask objects which cause a TypeError
when passed to the eager attention path:
TypeError: unsupported operand type(s) for +=: 'Tensor' and 'BlockMask'
This fix excludes GPT-OSS from using flex_attention, keeping it on
the eager path to avoid the BlockMask/Tensor type mismatch.
* Enable flex attention by default
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Avoid dropping flex attention when SDPA unsupported
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update rl_replacements.py
* Update rl.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update rl_replacements.py, remove chat template from codexes commits
* Update rl.py, got rid of gradient checkpointing code that did not work
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix torchvision compatibility check for source builds and future torch versions
The torchvision version check raised a hard ImportError for custom/source-built
PyTorch installations (e.g. AMD ROCm from source with +git* suffixes), even when
the actual build was functional. This also silently skipped any torch version
not already in the hardcoded table, giving no warning at all for future releases.
Changes:
- Detect custom/source builds by checking the raw version string's local
identifier against known standard prefixes (cu, rocm, cpu, xpu). Our custom
Version() strips local identifiers via regex, so detection must happen on the
raw string before parsing.
- Downgrade to a warning (instead of ImportError) for custom/source builds,
since their version numbers may not follow standard PyPI release pairings.
- Add formula-based inference for future torch versions not yet in the table.
The torch->torchvision minor version formula (torch 2.x -> tv 0.(x+15)) has
held for every release from torch 2.0 through 2.9. For formula-predicted
versions, mismatches produce a warning rather than a hard error.
- Add UNSLOTH_SKIP_TORCHVISION_CHECK=1 env var to skip the check entirely.
- Wrap importlib_version and Version calls in try/except so broken metadata
never crashes the import.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: stricter regex, case insensitivity, pre-release detection
Fixes three edge cases found during review:
1. Regex precision: cu/xpu now require a trailing digit (cu\d, xpu\d) to
avoid false negatives on suffixes like "+custom_build" that happen to
start with "cu". cpu/xpu match as exact strings only.
2. Case insensitivity: added re.IGNORECASE so "+ROCM6.3" and "+CPU" are
correctly recognized as standard builds rather than custom ones.
3. Pre-release detection: nightly/dev/alpha/beta/rc builds with standard
CUDA/ROCm suffixes (e.g. "2.7.0.dev20250301+cu124") now produce a
warning instead of a hard ImportError. These builds commonly have
version mismatches that are expected during development.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address PR review comments: fullmatch, env var casing, torchvision pre-release
1. Switch re.match to re.fullmatch for the custom build regex so the
entire local identifier must match. Fixes false negatives where
suffixes like +cu124_custom were misclassified as standard because
re.match only checked the start of the string.
2. Use .lower() for the UNSLOTH_SKIP_TORCHVISION_CHECK env var so
any casing of "true" / "TRUE" / etc. is accepted.
3. Check torchvision_version_raw for pre-release tags in addition to
torch_version_raw, so a stable torch paired with a nightly
torchvision (e.g. 0.23.0.dev...) also gets a warning instead of
a hard ImportError.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
