import_fixes: stub-module injection for peft.utils.transformers_weight_conversion on transformers 4.x (#5416)

* import_fixes: stub transformers.conversion_mapping so peft 0.19.x imports on transformers 4.x

patch_peft_weight_converter_compatibility currently opens with

    try:
        from peft.utils import transformers_weight_conversion as twc
    except (ImportError, AttributeError):
        return

which silently no-ops on (peft 0.19.x, transformers 4.57.x): peft's
transformers_weight_conversion module unconditionally imports two
transformers-v5 submodules at module top

    from transformers.conversion_mapping import ...
    from transformers.core_model_loading import ...

and neither submodule exists on transformers < 5. peft itself only USES
those submodules inside an is_transformers_ge_v5 branch, but the top of
file import still explodes with

    ModuleNotFoundError: No module named 'transformers.conversion_mapping'

The bare except above swallows that, so the weight converter compat
wrap never gets installed, and any downstream code that later does
from peft.utils import transformers_weight_conversion crashes with the
same ModuleNotFoundError.

Fix: synthesise minimal stub modules for transformers.conversion_mapping
and transformers.core_model_loading, install them into sys.modules, and
re-import peft.utils.transformers_weight_conversion so the kwargs compat
wrap can succeed on top. The stubs expose exactly the symbols peft 0.19.x
pulls in at module top (Concatenate / ConversionOps are real subclassable
classes since peft subclasses them as PeftConcatenate / FlattenDims /
PermuteDims), so peft's own class creation succeeds. None of the stubbed
callables actually fire on the 4.x branch because peft's runtime
is_transformers_ge_v5 gate keeps them unreachable.

Gating contract (strict no-op outside the (peft 0.19.x, transformers 4.x)
combination):
  * No-op if peft is not installed.
  * No-op if peft.utils.transformers_weight_conversion already imports
    clean (transformers v5+, or any peft fork off the v5 path).
  * Strictly additive: only stubs submodules that are currently missing
    from sys.modules / find_spec. We never overwrite the real
    transformers.conversion_mapping / transformers.core_model_loading
    on transformers v5+.
  * Idempotent: sentinel attribute (__unsloth_stub__) on the stub modules
    makes a second call return False, a third call return False, etc.
  * Surfaces drift unchanged: if peft fails for some reason OTHER than
    these two specific missing submodules, the original ImportError is
    left for the caller's own try/except to take over.

Forwards / backwards compatibility:
  * transformers 4.57.6 -> install stubs.
  * transformers 5.x (real submodules) -> first-import probe succeeds,
    return False, never touch sys.modules.
  * TRL 0.22 / 0.27 / 1.x -- none of these import either submodule
    directly; they reach the peft conversion module (if at all) through
    peft.tuners.tuners_utils, behind peft's own is_transformers_ge_v5
    gate. Stubs are therefore unreachable from TRL on a 4.x install,
    and on a 5.x install the real submodules win the import race.
  * peft 0.18 / 0.19 / 0.20 -- the symbols stubbed cover the union of
    what peft pulls at module top across the 0.19.x line; older peft
    that doesn't import the v5 submodules at all hits the cheap
    first-import-probe exit and we never touch sys.modules.

Wired into unsloth/_gpu_init.py to run BEFORE
patch_peft_weight_converter_compatibility (otherwise that function's
bare except would still silently no-op). Mirrors the equivalent fix
shipped in unsloth-zoo (the zoo-side stub installs itself via
apply_import_fixes() at zoo import time, but a user can run
unsloth without the zoo fix on an older unsloth_zoo, so the unsloth
side needs to own its own copy of the workaround).

tests/conftest.py is updated to pre-apply this specific fix via the
standalone import-fixes module so the GPU-free drift detector test
(tests/test_import_fixes_drift.py::test_peft_transformers_weight_conversion_importable_and_signature)
sees the same patched state that a real ``import unsloth`` would.
The pattern mirrors unsloth-zoo's tests/conftest.py
_apply_zoo_import_fixes_for_tests helper, scoped to just the peft fix.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

tests/conftest.py

@@ -139,3 +139,96 @@ if not _has_real_accelerator():
     if not _preload_device_type("unsloth"):
         _install_device_type_stub("unsloth.device_type")
     _patch_torch_cuda_for_import()
+
+
+# ---------------------------------------------------------------------------
+# Apply unsloth-local upstream-drift fixes that need to run before pytest
+# collects tests that import the affected third-party module directly.
+#
+# Specifically: ``from peft.utils import transformers_weight_conversion``
+# blows up on (peft 0.19.x + transformers 4.x) because peft unconditionally
+# imports two transformers-v5 submodules at module top. The production
+# import path applies the stub-injection workaround via
+# ``unsloth/_gpu_init.py``, but the GPU-free test harness above
+# deliberately avoids triggering the full ``unsloth`` package init (which
+# pulls in the CUDA / torch device chain). Load just the standalone
+# import-fixes module by file path so drift detectors that probe peft
+# see the same patched state a real unsloth install would.
+# ---------------------------------------------------------------------------
+
+
+def _apply_unsloth_peft_import_fix_for_tests() -> None:
+    import importlib.util as _ilu
+
+    try:
+        pkg_spec = _ilu.find_spec("unsloth")
+    except Exception:
+        return
+    if pkg_spec is None or not pkg_spec.submodule_search_locations:
+        return
+    fix_path = os.path.join(
+        pkg_spec.submodule_search_locations[0],
+        "import_fixes.py",
+    )
+    if not os.path.exists(fix_path):
+        return
+
+    mod_name = "unsloth.import_fixes"
+    _installed_skeleton = False
+    if mod_name in sys.modules:
+        mod = sys.modules[mod_name]
+    else:
+        # Submodule import requires SOME parent ``unsloth`` entry in
+        # sys.modules. Reuse one if a sibling conftest step already
+        # installed it (and don't pop in that case); otherwise install a
+        # bare skeleton and pop on the way out so subsequent
+        # ``import unsloth`` calls hit the real package init.
+        if "unsloth" not in sys.modules:
+            pkg = types.ModuleType("unsloth")
+            pkg.__path__ = list(pkg_spec.submodule_search_locations)
+            pkg.__spec__ = pkg_spec
+            pkg.__package__ = "unsloth"
+            pkg.__file__ = os.path.join(
+                pkg_spec.submodule_search_locations[0],
+                "__init__.py",
+            )
+            sys.modules["unsloth"] = pkg
+            _installed_skeleton = True
+        spec = _ilu.spec_from_file_location(mod_name, fix_path)
+        if spec is None or spec.loader is None:
+            if _installed_skeleton:
+                sys.modules.pop("unsloth", None)
+            return
+        mod = _ilu.module_from_spec(spec)
+        sys.modules[mod_name] = mod
+        try:
+            spec.loader.exec_module(mod)
+        except Exception:
+            sys.modules.pop(mod_name, None)
+            if _installed_skeleton:
+                sys.modules.pop("unsloth", None)
+            return
+
+    fix = getattr(mod, "fix_peft_transformers_weight_conversion_import", None)
+    if fix is None:
+        if _installed_skeleton:
+            sys.modules.pop("unsloth", None)
+        return
+    try:
+        fix()
+    except Exception:
+        # Individual fix is internally guarded; if the entry point itself
+        # blows up, don't take pytest collection down.
+        pass
+    finally:
+        # Drop our scratch skeleton so subsequent ``import unsloth``
+        # calls hit the real package init rather than our empty
+        # placeholder. The import-fixes module itself stays in
+        # sys.modules under ``unsloth.import_fixes`` -- python's import
+        # machinery is happy to find a submodule without an active
+        # parent entry.
+        if _installed_skeleton:
+            sys.modules.pop("unsloth", None)
+
+
+_apply_unsloth_peft_import_fix_for_tests()

unsloth/_gpu_init.py

@@ -153,6 +153,7 @@ from .import_fixes import (
     disable_torchcodec_if_broken,
     disable_broken_wandb,
     fix_trl_vllm_ascend,
+    fix_peft_transformers_weight_conversion_import,
     patch_peft_weight_converter_compatibility,
 )
 
@@ -177,6 +178,15 @@ patch_vllm_for_notebooks()
 patch_torchcodec_audio_decoder()
 disable_torchcodec_if_broken()
 disable_broken_wandb()
+# Must run BEFORE patch_peft_weight_converter_compatibility: on peft 0.19.x
+# + transformers 4.x, ``from peft.utils import transformers_weight_conversion``
+# raises ModuleNotFoundError because peft unconditionally imports
+# ``transformers.conversion_mapping`` and ``transformers.core_model_loading``
+# at module top, but neither exists on transformers <5. Stubbing those two
+# submodules first lets the converter compat patch actually wrap
+# ``build_peft_weight_mapping`` instead of silently no-opping in its bare
+# ``except (ImportError, AttributeError): return``.
+fix_peft_transformers_weight_conversion_import()
 patch_peft_weight_converter_compatibility()
 
 del fix_xformers_performance_issue
@@ -199,6 +209,7 @@ del patch_vllm_for_notebooks
 del patch_torchcodec_audio_decoder
 del disable_torchcodec_if_broken
 del disable_broken_wandb
+del fix_peft_transformers_weight_conversion_import
 del patch_peft_weight_converter_compatibility
 
 # Torch 2.4 has including_emulation

unsloth/import_fixes.py

@@ -1372,6 +1372,371 @@ def disable_broken_wandb():
         os.environ["WANDB_DISABLED"] = "true"
 
 
+# ---------------------------------------------------------------------------
+# peft 0.19.x  +  transformers 4.x  drift
+# ---------------------------------------------------------------------------
+#
+# peft 0.19.x ships ``peft/utils/transformers_weight_conversion.py`` with a
+# top-of-file ``from transformers.conversion_mapping import ...`` AND a
+# ``from transformers.core_model_loading import ...``. Neither submodule
+# exists on transformers < 5.x. The peft module's own header is explicit
+# ("don't import from this module unless transformers v5+ is used"), and
+# peft itself only triggers the import at RUNTIME inside an
+# ``if is_transformers_ge_v5:`` branch (``peft/tuners/tuners_utils.py``).
+# However any code that does the obvious
+# ``from peft.utils import transformers_weight_conversion`` -- including
+# Unsloth's own ``patch_peft_weight_converter_compatibility`` below
+# (which wraps ``build_peft_weight_mapping``) -- still tries to import the
+# module unconditionally and explodes with
+#
+#     ModuleNotFoundError: No module named 'transformers.conversion_mapping'
+#
+# on the 4.x stack. The bare ``except (ImportError, AttributeError)`` guard
+# inside ``patch_peft_weight_converter_compatibility`` then catches that
+# and silently no-ops, leaving downstream consumers to crash later with
+# the same ModuleNotFoundError the first time anything imports
+# ``peft.utils.transformers_weight_conversion``.
+#
+# Fix: when (and only when) the import is broken AND the underlying
+# transformers really is missing those two submodules, inject minimal stub
+# modules into ``sys.modules`` with exactly the symbols peft pulls in at
+# its module top. The stubs are dead inert on transformers 4.x because
+# peft never calls into them on that branch (its own ``is_transformers_ge_v5``
+# gate keeps them unreachable at runtime).
+#
+# On transformers v5+, both submodules exist for real, this function is a
+# strict no-op (the first-import probe passes and we return immediately)
+# and we never touch ``sys.modules``.
+# ---------------------------------------------------------------------------
+
+# Sentinel attribute stamped on stub modules so a second call is a strict
+# no-op and so third parties can introspect ``__unsloth_stub__`` to detect
+# our patch.
+_UNSLOTH_STUB_SENTINEL = "__unsloth_stub__"
+
+
+def _peft_stub_module_importable(name):
+    """True iff ``import {name}`` would succeed without ImportError.
+
+    Uses ``find_spec`` rather than a raw ``import`` to avoid triggering
+    arbitrary module-level side effects when we're only probing. Also
+    treats an already-cached ``sys.modules`` entry as importable.
+    """
+    if name in sys.modules and sys.modules[name] is not None:
+        return True
+    try:
+        return importlib.util.find_spec(name) is not None
+    except (ImportError, ValueError, ModuleNotFoundError):
+        return False
+
+
+def _make_peft_stub_module(fullname):
+    """Create a fresh stub module marked with our sentinel."""
+    import types as _types
+
+    mod = _types.ModuleType(fullname)
+    mod.__file__ = f"<unsloth stub: {fullname}>"
+    mod.__package__ = fullname.rpartition(".")[0]
+    setattr(mod, _UNSLOTH_STUB_SENTINEL, True)
+    return mod
+
+
+def _install_transformers_conversion_mapping_stub():
+    """Synthesise a ``transformers.conversion_mapping`` module.
+
+    Provides exactly the three symbols peft 0.19.x imports at module top:
+
+    * ``_MODEL_TO_CONVERSION_PATTERN`` -- a real ``dict`` (peft calls
+      ``.copy()`` on it at module top and then does keyed assignment).
+    * ``get_checkpoint_conversion_mapping(model_type)`` -- returns
+      ``None`` (i.e. "no v5 conversion registered for this model type").
+      peft only invokes this at runtime inside
+      ``convert_peft_config_for_transformers`` /
+      ``convert_peft_adapter_state_dict_for_transformers``, and both
+      early-return on ``None``.
+    * ``get_model_conversion_mapping(model)`` -- returns ``None``. Same
+      runtime guard story.
+
+    On transformers 4.x peft's own gate (``is_transformers_ge_v5``) means
+    these callables never actually fire, but we make them well-behaved
+    just in case some caller invokes them directly.
+    """
+    name = "transformers.conversion_mapping"
+    existing = sys.modules.get(name)
+    if existing is not None and getattr(existing, _UNSLOTH_STUB_SENTINEL, False):
+        return existing
+
+    mod = _make_peft_stub_module(name)
+
+    # peft does ``_MODEL_TO_CONVERSION_PATTERN = _MODEL_TO_CONVERSION_PATTERN.copy()``
+    # at module top, then keyed assignment. A real dict is sufficient.
+    mod._MODEL_TO_CONVERSION_PATTERN = {}
+
+    def get_checkpoint_conversion_mapping(model_type, *args, **kwargs):
+        # ``None`` is peft's "no conversion registered" sentinel; both
+        # callsites in peft early-return on it.
+        return None
+
+    def get_model_conversion_mapping(model, *args, **kwargs):
+        # Same story: peft treats ``None`` / empty list as "nothing to do".
+        return None
+
+    mod.get_checkpoint_conversion_mapping = get_checkpoint_conversion_mapping
+    mod.get_model_conversion_mapping = get_model_conversion_mapping
+
+    sys.modules[name] = mod
+    # Attach to the parent package as well so ``import transformers;
+    # transformers.conversion_mapping`` works just like a real submodule.
+    parent = sys.modules.get("transformers")
+    if parent is not None and not hasattr(parent, "conversion_mapping"):
+        try:
+            parent.conversion_mapping = mod
+        except Exception:
+            # Defensive: a frozen / read-only parent still leaves the
+            # sys.modules entry in place, which is enough for
+            # ``from transformers.conversion_mapping import ...``.
+            pass
+    return mod
+
+
+def _install_transformers_core_model_loading_stub():
+    """Synthesise a ``transformers.core_model_loading`` module.
+
+    Provides the eight symbols peft 0.19.x imports at module top:
+
+      Classes: ``ConversionOps``, ``Concatenate``, ``MergeModulelist``,
+      ``Transpose``, ``WeightConverter``, ``WeightRenaming``.
+
+      Callables: ``dot_natural_key``, ``rename_source_key``.
+
+    Peft subclasses ``Concatenate`` and ``ConversionOps`` at module top
+    (``PeftConcatenate``, ``FlattenDims``, ``PermuteDims``), so those two
+    MUST be real classes -- not callables, not ``object()`` -- or class
+    creation will fail at import. The remaining classes only appear in
+    ``isinstance`` checks / runtime construction calls that are gated
+    behind ``is_transformers_ge_v5`` upstream and never fire on the 4.x
+    branch, but we still make them real classes so any third party that
+    does ``from transformers.core_model_loading import WeightConverter``
+    after this patch sees a sensible (if inert) class.
+    """
+    name = "transformers.core_model_loading"
+    existing = sys.modules.get(name)
+    if existing is not None and getattr(existing, _UNSLOTH_STUB_SENTINEL, False):
+        return existing
+
+    mod = _make_peft_stub_module(name)
+
+    class ConversionOps:
+        """Stub base class. Subclassing is permitted (peft does this)."""
+
+        def convert(self, *args, **kwargs):  # pragma: no cover - inert stub
+            raise NotImplementedError(
+                "unsloth stub: transformers.core_model_loading.ConversionOps "
+                "is a no-op on transformers <5. Upgrade transformers to v5+ "
+                "to use peft.utils.transformers_weight_conversion at runtime."
+            )
+
+        @property
+        def reverse_op(self):  # pragma: no cover - inert stub
+            raise NotImplementedError
+
+    class Concatenate(ConversionOps):
+        """Stub. Peft subclasses this as ``PeftConcatenate``."""
+
+        def __init__(self, dim = 0, *args, **kwargs):
+            self.dim = dim
+
+    class MergeModulelist(ConversionOps):
+        """Stub. Peft only uses this for ``isinstance(op, MergeModulelist)``."""
+
+        def __init__(self, *args, **kwargs):
+            pass
+
+    class Transpose(ConversionOps):
+        """Stub. Peft instantiates ``Transpose(dim0=0, dim1=1)`` at runtime."""
+
+        def __init__(self, dim0 = 0, dim1 = 1, *args, **kwargs):
+            self.dim0 = dim0
+            self.dim1 = dim1
+
+    class WeightConverter:
+        """Stub. Peft uses for ``isinstance`` and runtime construction."""
+
+        def __init__(self, *args, **kwargs):
+            # Accept any signature: peft's real upstream class evolves.
+            self.args = args
+            self.kwargs = kwargs
+
+    class WeightRenaming:
+        """Stub. Peft instantiates ``WeightRenaming(source, target)``."""
+
+        def __init__(
+            self,
+            source_patterns = None,
+            target_patterns = None,
+            *args,
+            **kwargs,
+        ):
+            # Support both positional and keyword forms.
+            self.source_patterns = source_patterns
+            self.target_patterns = target_patterns
+
+    def dot_natural_key(key):
+        """Stub key function. Peft only calls this inside a v5-gated path."""
+        return key
+
+    def rename_source_key(original_key, renamings, converters):
+        """Stub. Returns ``(original_key, None)`` -- v5-gated upstream."""
+        return original_key, None
+
+    mod.ConversionOps = ConversionOps
+    mod.Concatenate = Concatenate
+    mod.MergeModulelist = MergeModulelist
+    mod.Transpose = Transpose
+    mod.WeightConverter = WeightConverter
+    mod.WeightRenaming = WeightRenaming
+    mod.dot_natural_key = dot_natural_key
+    mod.rename_source_key = rename_source_key
+
+    sys.modules[name] = mod
+    parent = sys.modules.get("transformers")
+    if parent is not None and not hasattr(parent, "core_model_loading"):
+        try:
+            parent.core_model_loading = mod
+        except Exception:
+            pass
+    return mod
+
+
+def fix_peft_transformers_weight_conversion_import():
+    """Make ``from peft.utils import transformers_weight_conversion`` work.
+
+    On any (peft 0.19.x, transformers 4.x) pair the import otherwise fails
+    with ``ModuleNotFoundError: No module named 'transformers.conversion_mapping'``
+    because the peft module unconditionally imports two transformers v5
+    submodules even though peft itself only USES them inside an
+    ``if is_transformers_ge_v5:`` branch. See the block comment above for
+    details.
+
+    Must run BEFORE ``patch_peft_weight_converter_compatibility``: the
+    latter wraps ``twc.build_peft_weight_mapping`` and its bare
+    ``except (ImportError, AttributeError): return`` would silently
+    no-op on the unfixed import, leaving downstream consumers to crash
+    later with the same ModuleNotFoundError.
+
+    Gating contract:
+      * No-op if ``peft`` is not installed.
+      * No-op if ``transformers`` is not installed (unfixable -- the
+        real symptom would be a different ImportError on the very
+        first ``import peft``).
+      * No-op if ``peft.utils.transformers_weight_conversion`` already
+        imports cleanly (transformers v5+, or a peft fork that uses
+        non-v5 paths).
+      * Idempotent: a second call sees our sentinel-stamped stubs and
+        returns immediately.
+      * Strictly additive: only installs a stub for a transformers
+        submodule that is currently MISSING. We never overwrite a real
+        ``transformers.conversion_mapping`` /
+        ``transformers.core_model_loading`` module on transformers v5+.
+
+    Forwards / backwards compatibility:
+      * transformers 4.57.x (no submodule) -> install stubs.
+      * transformers 5.x (real submodule) -> first-import succeeds, return.
+      * TRL 0.22 / 0.27 / 1.x -- these don't import either submodule
+        directly; they reach the peft conversion module (if at all)
+        through ``peft.tuners.tuners_utils``, behind peft's own
+        ``is_transformers_ge_v5`` gate. Our stubs are therefore
+        unreachable from TRL on a 4.x install, and on a 5.x install the
+        real submodules win the import race against our patch.
+
+    Returns ``True`` if the patch was applied (or had been applied
+    previously), ``False`` if no action was needed, ``None`` if peft is
+    not installed.
+    """
+    # 1. Cheap exit: no peft installed.
+    if importlib.util.find_spec("peft") is None:
+        return None
+
+    # 2. Cheap exit: peft.utils.transformers_weight_conversion already
+    #    importable -- either we already stubbed and re-imported, or
+    #    transformers is v5+ with real submodules. Try once and return
+    #    on success.
+    try:
+        importlib.import_module("peft.utils.transformers_weight_conversion")
+        return False
+    except ModuleNotFoundError as exc:
+        # Only act on our specific drift class. Anything else surfaces
+        # the original exception on the next import attempt.
+        missing = getattr(exc, "name", "") or ""
+        if missing not in (
+            "transformers.conversion_mapping",
+            "transformers.core_model_loading",
+        ):
+            return False
+    except ImportError as exc:
+        # Older Python only raises ImportError without `.name`, so also
+        # string-match the message for our specific drift.
+        msg = str(exc)
+        if (
+            "transformers.conversion_mapping" not in msg
+            and "transformers.core_model_loading" not in msg
+        ):
+            return False
+
+    # 3. Confirm transformers is loaded; if not, try to load it so our
+    #    stub modules can be attached to the parent package. If that
+    #    fails the user's stack is too broken for us to repair.
+    transformers_root = sys.modules.get("transformers")
+    if transformers_root is None:
+        try:
+            transformers_root = importlib.import_module("transformers")
+        except Exception:
+            return False
+
+    # 4. Stub only the submodules that are genuinely missing. We do NOT
+    #    stub a module that already exists for real -- that would
+    #    clobber correct behaviour on transformers v5+.
+    patched_any = False
+    if not _peft_stub_module_importable("transformers.conversion_mapping"):
+        _install_transformers_conversion_mapping_stub()
+        patched_any = True
+
+    if not _peft_stub_module_importable("transformers.core_model_loading"):
+        _install_transformers_core_model_loading_stub()
+        patched_any = True
+
+    if not patched_any:
+        # Both real submodules already exist -- ``transformers_weight_conversion``
+        # must have failed for some other reason. Bail; the next import
+        # attempt will surface the original exception unchanged.
+        return False
+
+    # 5. Force the peft module through a fresh import now that the
+    #    stubs are in place. If a previous failed import left a ``None``
+    #    cache entry in ``sys.modules`` we have to drop it so importlib
+    #    will retry.
+    pkg = "peft.utils.transformers_weight_conversion"
+    if pkg in sys.modules and sys.modules[pkg] is None:
+        del sys.modules[pkg]
+    try:
+        importlib.import_module(pkg)
+    except Exception:
+        # If even with the stub the module won't import (some other
+        # upstream API drift) we swallow. Callers using
+        # ``try / except (ImportError, AttributeError)`` will take over.
+        # Crucially the stubs stay installed so the NEXT import attempt
+        # (after whatever transient condition clears) still succeeds.
+        return True
+
+    logger.info(
+        "Unsloth: stubbed transformers.conversion_mapping / "
+        "transformers.core_model_loading so peft.utils."
+        "transformers_weight_conversion imports cleanly on "
+        "transformers <5."
+    )
+    return True
+
+
 def patch_peft_weight_converter_compatibility():
     """Allow PEFT converter rebuilds on legacy converter constructors."""
     try: