* packing optimziation with cache to reduce D2H copy
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* cache per device to avoid race condition for multi-gpu
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* add cache freeing up func
---------
Co-authored-by: ruixiangw <ruixiangw@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: ruixiang <wangruixiang07@outlook.com>
* Rebuild Studio branch on top of main
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix security and code quality issues for Studio PR #4237
- Validate models_dir query param against allowed directory roots
to prevent path traversal in /api/models/local endpoint
- Replace string startswith() with Path.is_relative_to() for
frontend path traversal check in serve_frontend
- Sanitize SSE error messages to not leak exception details to
clients (4 locations in inference.py)
- Bind port-discovery socket to 127.0.0.1 instead of all interfaces
in llama_cpp backend
- Import datasets_root and resolve_output_dir in embedding training
function to fix NameError and use managed output directory
- Remove stale .gitignore entries for package-lock.json and test
directories so tests can be tracked in version control
- Add venv-reexecution logic to ui CLI command matching the studio
command behavior
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move models_dir path validation before try/except block
The HTTPException(403) was inside the try/except Exception handler,
so it would be caught and re-raised as a 500. Moving the validation
before the try block ensures the 403 is returned directly and also
makes the control flow clearer for static analysis (path is validated
before any filesystem operations).
* Use os.path.realpath + startswith for models_dir validation
CodeQL py/path-injection does not recognize Path.is_relative_to() as
a sanitizer. Switched to os.path.realpath + str.startswith which is
a recognized sanitizer pattern in CodeQL's taint analysis. The
startswith check uses root_str + os.sep to prevent prefix collisions
(e.g. /app/models_evil matching /app/models).
* Never pass user input to Path constructor in models_dir validation
CodeQL traces taint through Path(resolved) even after a startswith
barrier guard. Fix: the user-supplied models_dir is only used as a
string for comparison against allowed roots. The Path object passed
to _scan_models_dir comes from the trusted allowed_roots list, not
from user input. This fully breaks the taint chain.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Refactor loss computation to include completion_mask
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fixes for trl 0.28 and above
Remove sync/reload weights calls , remove vllm.LLM instantiation
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Refactor loss computation to include completion_mask
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fixes for trl 0.28 and above
Remove sync/reload weights calls , remove vllm.LLM instantiation
* patch rpc in openenv for newer trl
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pluesclues <136766175+pluesclues@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix gpt temporary patch for grpo to happen after compile
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Refactor loss computation to include completion_mask
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
trl/trainer/callbacks.py imports is_wandb_available from
accelerate.utils, not from transformers. The original fix in #4147
only patched the transformers version, so `from trl import GRPOTrainer`
still crashed via the callbacks.py -> accelerate -> wandb path.
Must patch both the source module (accelerate.utils.imports) AND the
re-export namespace (accelerate.utils) since Python's
`from accelerate.utils import X` reads from the latter, which holds
its own cached reference.
* Fix broken wandb import crashing unsloth startup
When wandb is installed but broken (e.g., wandb < 0.19.11 with
protobuf >= 6.0), the import chain unsloth -> trl -> transformers ->
is_wandb_available() -> import wandb crashes with:
ImportError: cannot import name 'Imports' from
'wandb.proto.wandb_telemetry_pb2'
This happens because transformers' is_wandb_available() has no
try/except around `import wandb`. The error propagates up and kills
`from unsloth import FastLanguageModel` even though wandb is optional.
Add disable_broken_wandb() following the same pattern as
disable_torchcodec_if_broken(). It proactively tries importing wandb
during early init, and if the import fails, patches
is_wandb_available() to return False and sets WANDB_DISABLED=true.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fixup mapper issues and resolve properly
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: update GGUF save paths to use ~/.unsloth/llama.cpp with Windows support
* fix: quote LLAMA_CPP_DEFAULT_DIR in fallback shell commands to handle paths with spaces
* refactor: deduplicate platform-specific build instructions in quantization error message
* chore: remove accidentally committed PR description file
* Fix import safety and f-string bugs in save.py
- H4: Add defensive try/except for LLAMA_CPP_DEFAULT_DIR and IS_WINDOWS imports
with fallback defaults, so save.py works even if zoo PR #526 is not merged yet
- H5: Fix Kaggle error path using plain "Error: {e}" instead of f"Error: {e}",
so the actual exception is shown to users
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix lm_head lora save
* Fix _need_to_train_embeddings guard for lm_head LoRA targets
When lm_head is already in final_modules as a LoRA target, the
_need_to_train_embeddings block should not also add it to
modules_to_save. This prevents dual-wrapping (LoRA + modules_to_save
on the same module) which causes assertion failures downstream.
Check if embed_tokens/lm_head are already being trained as LoRA
targets before adding them to modules_to_save. Also prevents
duplicate entries with elif guards.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Current arch.startswith("gfx1") incorrectly matches:
- RDNA1 (gfx10xx) and RDNA2 (gfx103x): not ROCm supported
- gfx1102 (RX 7600), gfx1103 (Phoenix APU): not in ROCm support matrix
- gfx1150/1151/1152 (RDNA3.5 APUs): not in ROCm support matrix
Replace with explicit whitelist aligned to the ROCm Linux support matrix:
https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/system-requirements.html
gfx1100 - RDNA3 discrete (RX 7900 series, PRO W7900/W7800)
gfx1101 - RDNA3 discrete (RX 7800/7700 series, PRO W7700)
gfx1200 - RDNA4 discrete (RX 9060 series)
gfx1201 - RDNA4 discrete (RX 9070 series, AI PRO R9700)
Mirrors the existing is_cdna() pattern. Avoids silently applying
unverified Triton kernel tuning to unsupported hardware.
Fix global dequantize buffer dtype mismatch when loading multiple 4-bit models with different dtypes in the same process. Adds dtype check alongside existing None check for WEIGHT_BUFFER in both CUDA/HIP and XPU paths.
Use 16 warps for RDNA in the chunked cross-entropy forward kernel
(large vocab > 65536), matching the existing CDNA optimization.
Benchmarked on W7900 (gfx1100) with actual unsloth kernels (5 trials, median):
- Chunked CE forward (BS=65536): 16 warps = 2.4-2.6x faster than 32
- All other kernels (LayerNorm, RoPE, SwiGLU): default heuristic is
already optimal for RDNA; no modification needed.
Depends on: #4109 (provides is_rdna() detection)
TMA (Tensor Memory Accelerator) is an NVIDIA Hopper+ feature that does
not exist on AMD GPUs. However, _check_tma_support() incorrectly
returns True on ROCm because:
1. torch.cuda.get_device_capability() returns (11, 0) for gfx1100,
satisfying the >= 9 check intended for Hopper (sm_90).
2. ROCm Triton exports tl.make_tensor_descriptor (the symbol exists
even though the hardware does not support TMA).
This would cause MoE grouped_gemm to attempt TMA operations on AMD
GPUs, leading to runtime failures.
Fix: early-return False for HIP devices, matching the existing XPU
guard.
* fix(Triton): ensure float32 eps in RMS LayerNorm rsqrt for HIP/ROCm
On HIP (AMD ROCm), Triton constexpr eps may not promote to float32
in rsqrt, causing numerical instability (NaN/Inf) on RDNA GPUs
(gfx1100, gfx1151 Strix Halo, etc.).
Use tl.full((), eps, tl.float32) to explicitly create a float32
scalar before adding to row_var in rsqrt. Applied to both standard
and Gemma RMS LayerNorm forward kernels.
Tested on W7900 (gfx1100): full test suite passed (dim 512-2048,
bf16/fp16, various seqlen).
Related: #3385, #3588
* Apply same float32 eps fix to layernorm.py for PR #4110
layernorm.py has the identical tl.constexpr eps pattern in
layernorm_forward that can misfire on HIP/ROCm. Apply the same
tl.full((), eps, tl.float32) fix for consistency.
Both testing_suite_layernorm (standard LayerNorm) and
testing_suite_layernorm (RMS LayerNorm) pass on NVIDIA after
this change.
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix(ROCm): comprehensive RDNA GPU support - fix Gemma3 NaN & add is_rdna()
- Add is_rdna() detection for RDNA3/3.5/RDNA4 consumer GPUs (gfx11xx, gfx1151, gfx12xx)
- Disable torch.compile for Gemma3 on HIP to fix NaN loss (fixes#3385, #4029)
- Export is_cdna/is_rdna from kernels for downstream use
- Import is_rdna into cross_entropy_loss for future RDNA-specific tuning
Tested on AMD Radeon PRO W7900 (gfx1100) with ROCm 7.1:
✓ Gemma3-1B: loss 3.37→3.25 (no NaN)
✓ Llama-3.2-1B: loss 2.44→2.37 (no NaN)
✓ Qwen2.5-1.5B: loss 1.89→1.85 (no NaN)
✓ RMS LayerNorm Triton kernel: bf16/fp16 PASSED
✓ Cross Entropy Loss Triton kernel: 32K/256K vocab PASSED
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: scope compile disable to RDNA only, use partial mode, remove unused import
Changes based on Daniel's review:
1. (HIGH) Replace DEVICE_TYPE=='hip' with is_rdna() to avoid disabling
torch.compile on CDNA GPUs (MI250X/MI300X/MI350) where it works fine
2. (MEDIUM) Use 'partial' instead of '1' for UNSLOTH_COMPILE_DISABLE to
only disable model forward compilation while keeping loss compilation,
matching the existing Sesame pattern
3. (LOW) Remove unused is_rdna import from cross_entropy_loss.py (F401)
* Remove redundant is_cdna/is_rdna exports from kernels/__init__.py
These functions are imported directly from .utils where needed
(e.g. cross_entropy_loss.py, loader.py). No external code imports
them from the unsloth.kernels namespace.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The function (introduced in #3923) assumed that the absence of
`triton.runtime.triton_key` on ROCm means torch.compile will crash.
Investigation shows this is incorrect:
1. `triton.runtime.triton_key` was renamed/removed in the ROCm Triton
fork — it does not exist at that path. However,
`triton.compiler.compiler.triton_key` (the path torch._inductor
actually imports) EXISTS and works correctly on ROCm.
2. Both call-sites in torch._inductor (codecache.py and
async_compile.py) already wrap the import in try/except, so even a
genuinely missing triton_key would be handled gracefully.
3. Comprehensive testing on ROCm 7.1 + Triton 3.4.0 + gfx1100 confirms
torch.compile works correctly for matmul, cross-entropy, RMSNorm,
multi-layer transformer forward+backward, and LoRA — all without
triton.runtime.triton_key.
The original code was also ineffective (environment variables set after
torch import have no effect on torch._dynamo config), so removing it
has zero behavioral change on existing installations.
Supersedes the compile-disable portion of #3923.
* Fix transformers v5 RoPE inv_freq corruption during model loading
Transformers v5 initializes models on the meta device, then
_move_missing_keys_from_meta_to_device() replaces all non-persistent
buffers with torch.empty_like() (uninitialized memory). Vanilla
transformers restores inv_freq via _init_weights() checking for
original_inv_freq, but Unsloth's LlamaRotaryEmbedding subclasses
lack this attribute, so inv_freq stays corrupted with garbage values.
This caused 5-11x higher training loss on transformers v5 for all
models using Unsloth's rope (Llama 3.x, Qwen3, Mistral, TinyLlama,
Granite). Models using native transformers rope (Gemma, Phi-4,
Falcon-H1) were unaffected.
The fix recomputes inv_freq from the stored base/dim after model
loading, applies model-specific scaling via _apply_inv_freq_scaling(),
and rebuilds cos/sin caches. Also handles LongRopeRotaryEmbedding
(Phi-3.5 style short/long inv_freq). Guarded by transformers >= 5.0.0
so it is a no-op on v4.
Tested on: Llama 3.1 8B, Llama 3.2 3B, Qwen3 14B, Qwen3 4B, Phi-4,
TinyLlama, Mistral 7B, Gemma2 2B, Falcon-H1 -- all v5 losses now
match v4 baselines to < 0.004 absolute difference.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Unpack BatchEncoding in generate() for v4/v5 backwards compatibility
Old notebooks pass the full tokenizer output as input_ids:
inputs = tokenizer(..., return_tensors="pt").to("cuda")
model.generate(input_ids=inputs, ...)
This worked on transformers v4 because generate() internally
extracted the tensor. Transformers v5 calls .shape on input_ids
directly, which crashes since BatchEncoding has no .shape attribute.
Fix: in unsloth_fast_generate(), detect when input_ids is a dict-like
object (BatchEncoding) and unpack its contents into separate kwargs
before forwarding to the underlying generate(). This makes both old
and new notebook patterns work on both v4 and v5.
* Remove redundant seen_ids dedup in _fix_rope_inv_freq
named_modules() already deduplicates with remove_duplicate=True (default).
Also clarify that native v5 rotary classes (Gemma3 etc.) have original_inv_freq
which transformers v5's _init_weights() uses to restore inv_freq, so they do
not need this fix.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix left-padding masks and positions in batched decode/prefill
* Fix batched generation with left padding
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix attention mask handling, padding_idx zeroing, and Mistral batched generation
1. attention_dispatch.py: Fall back from flash/xformers to SDPA when an
attention_mask is present, since flash attention only supports causal
masking via flag and cannot consume arbitrary padding masks.
2. gemma2.py: Apply attention_mask during decode inference for bsz > 1.
Guard against boolean SWA/GA flags with isinstance check. Slice mask
to match K/V length when sliding window is active. Remove dead
commented-out SDPA branch (SDPA does not support softcapping).
3. granite.py: Apply attention_mask during decode inference for bsz > 1.
Remove dead commented-out SDPA branch and misleading comment.
4. mistral.py: Fix 2D-to-4D padding mask conversion -- convert 0/1 mask
to additive format (0 for keep, -inf for mask) before combining with
the causal mask. Force SDPA backend when attention_mask is present.
5. llama.py: Skip zeroing embed_tokens.weight[padding_idx] when the
embedding is weight-tied to lm_head, since zeroing the shared weight
forces logit(pad) = 0 which is higher than real token logits in models
like Gemma, causing the decoder to emit pad tokens as gibberish. Also
add eos != pad guard, clean up unused _seq_length variable, and fix
get_max_cache_shape handling.
6. vision.py: Same padding_idx fix as llama.py for the vision model
loading path.
Tested on gemma-2b-it, gemma-2-2b-it, Llama-3.2-1B, Mistral-7B-v0.3,
Qwen2.5-0.5B, Qwen3-0.6B with flash-attn 2.8.3 active. All outputs
coherent, zero crashes, zero resize warnings.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Inference path optimizations: eliminate per-layer GPU-CPU sync, cache inspect.signature, add Granite SDPA split
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* More inference path optimizations across model files
- gemma: hoist rotary_seq_len computation to model level (eliminates N
per-layer GPU-CPU syncs from position_ids.max().item()), pre-convert
attention mask to bool once for all layers, use scalar float multiply
instead of torch.tensor allocation for embedding scaling
- gemma2: use in-place tanh_() for softcap attention, use scalar float
multiply for embedding scaling
- granite: pre-convert attention mask to bool once for all layers
- cohere: use in-place neg_() for rotary embedding (consistent with
all other model files)
- falcon_h1: use in-place mul_() for key_multiplier scaling
- llama: use in-place tanh_() for logit softcapping
* Revert scalar multiply for Gemma/Gemma2 embedding scaling
The original torch.tensor(..., dtype=hidden_states.dtype) is intentional:
sqrt(3072) rounds to 55.5 in bfloat16 vs 55.4256 in float32. A plain
scalar multiply may compute at higher precision internally, producing
different results. Restore the explicit dtype-cast tensor to match the
training path in LlamaModel_fast_forward.
* Fix hardcoded cuda:0 device strings and add Cohere .eq(0) bool mask
Replace 15 hardcoded "cuda:0" with f"{DEVICE_TYPE_TORCH}:0" across
gemma.py, gemma2.py, cohere.py, and falcon_h1.py to support multi-GPU
and non-CUDA devices (XPU, etc.). Add .eq(0) bool mask pre-conversion
in CohereModel_fast_forward_inference for batched inference consistency
with llama.py, granite.py, and gemma.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Disable flex_attention for Mllama (Llama 3.2 Vision)
Mllama's _update_causal_mask uses the deprecated make_flex_block_causal_mask
which creates a BlockMask with Q_LEN=KV_LEN=total_seq_len. During decode
with KV cache, q_len=1 but the block_mask still has Q_LEN=total_seq_len,
causing a ValueError. This is an upstream transformers issue -- newer models
use flex_attention_mask from masking_utils which handles decode correctly
via cache_position, but mllama has not been updated yet.
Add mllama to the exclusion list in prefer_flex_attn_if_supported alongside
gpt_oss so it falls back to sdpa, which works correctly for both training
and inference.
* Fix off-by-one in sliding window K/V slicing for gemma2, qwen3, falcon_h1, cohere
The old formula `slicing_tokens = 1 - sliding_window` uses negative indexing
that keeps `sliding_window - 1` tokens instead of `sliding_window`. For example
with sliding_window=32 and kv_seq_len=100, `1-32 = -31` keeps indices 69..99
(31 tokens) instead of the correct 68..99 (32 tokens).
Replace with `start = kv_seq_len - sliding_window` to match the fix already
applied in llama.py and the canonical definition in transformers masking_utils
(sliding_window_overlay: kv_idx > q_idx - W, which keeps exactly W tokens).
Also add attention_mask slicing after K/V trim in qwen3, falcon_h1, and cohere
to prevent mask/K dimension mismatch during batched SDPA inference, matching
the pattern already used in llama.py.
Currently only gemma2 (sliding_window=4096) is actively affected. The other
three models have sliding_window=None in their configs so the code path is
not triggered, but this keeps it correct for any future models that set it.
* Fix Gemma2 softcapping order: apply mask after softcap, not before
The attention mask must be applied AFTER logit softcapping, not before.
Both the Google DeepMind reference implementation (google-deepmind/gemma,
gm/nn/_modules.py lines 254-277) and transformers' eager_attention_forward
(gemma2/modeling_gemma2.py lines 187-193) use this order:
1. logits = Q @ K^T * scale
2. logits = tanh(logits / softcap) * softcap # softcap first
3. logits = logits + mask # mask after
4. probs = softmax(logits)
The PR had the mask addition before softcapping, which causes tanh to
clamp the -inf mask values to -softcap instead of preserving them as -inf
for softmax. While the practical impact is small (masked positions get
~1e-23 probability instead of exact zero), this should match upstream.
* Clarify GQA condition precedence and remove stale comments
Add explicit parentheses to grouped query attention conditions in
llama.py, qwen3.py, granite.py to make operator precedence clear.
The expression `bsz == 1 or not X and Y` relies on Python binding
`not` > `and` > `or` which is correct but easy to misread.
Remove dead commented-out code (`# else: # Knn, Vnn = Knn, Vnn`)
and stale mask comments (`# if attention_mask ...`) from the bsz==1
fast path in llama, qwen3, cohere, falcon_h1, gemma2 inference
functions. These were leftover from the pre-batched-inference
structure and no longer apply.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Allow fp8 for non fast inference
* Extensive fp8 alow and quantizer patch
* Clean up commented-out code, duplicate import, and revert unnecessary Version() changes
- Delete commented-out FP8 fast_inference guard in FastModel (loader.py)
instead of leaving it commented -- matches FastLanguageModel which was
properly deleted
- Delete commented-out fast_inference guard in loader_utils.py
- Remove duplicate `from transformers import GenerationConfig, CompileConfig`
in vision.py (line 112 already imports both plus AutoConfig)
- Revert Version(trl.__version__) back to Version(trl) in trainer.py --
trainer.py imports Version from unsloth_zoo.utils which already handles
module objects
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Add resilience to TRL internal API reclassification
TRL is moving toward v1.0 and will reclassify several
currently-importable symbols as internal with no stability
guarantees. This adds try/except cascading imports with local
fallbacks so Unsloth keeps working regardless of whether TRL
removes, moves, or restructures these symbols.
Changes:
- rl.py: Add try/except cascade for unwrap_model_for_generation
with local contextmanager fallback. Wire sanitize_logprob from
RL_REPLACEMENTS into the compiled trainer template (same pipeline
as selective_log_softmax and other global functions). Add import
math and import logging to the template header.
- rl_replacements.py: Remove inline import of sanitize_logprob
from trl.scripts.vllm_serve in the regex replacement. The
function is now a module-level global in the compiled file.
- tokenizer_utils.py: Wrap dynamic exec import with per-item
fallback so a single removed symbol does not break the entire
bulk import.
Depends on unslothai/unsloth-zoo#516.
Tested across all TRL versions from 0.22.2 through 0.29.0.dev0
(git main). Training losses and grad norms are bit-identical
to unpatched runs.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
