feat: support text-only loading of Gemma 3 27B via FastLanguageModel (skip SiglipVisionModel) (#5816)

* feat: support text-only loading of Gemma 3 27B via FastLanguageModel (skip SiglipVisionModel)

* test: instantiate text-only Gemma3 model and assert no vision tower

Existing tests were AST source-introspection plus a config-resolves-to-
text-config check; none actually instantiated a model from the
text-only config. Add a small integration test that builds a shrunken
Gemma3TextConfig (CPU-cheap), instantiates the matching CausalLM
class, and asserts the resulting model exposes the LM head and has no
vision_tower or multi_modal_projector attribute.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Deduplicate _get_text_only_config into _utils for PR #5816

* Fall back to full model when a VLM has no text-only class for PR #5816

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Preserve quantization_config and clarify warning for text-only loading for PR #5816

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Only take text-only path when the VLM has its own text decoder for PR #5816

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Convert source string-match assertions to AST checks per Gemini review

* Load real VLM text weights on transformers 5.x for text-only mode in PR #5816

transformers >=5 changed Gemma3ForCausalLM base_model_prefix from language_model to model, so a VLM checkpoint's text weights (gemma3: language_model.model.*, gemma3n: model.language_model.*) no longer auto-strip onto the text decoder and were silently initialized random. Add a version-gated key_mapping that remaps them onto the text keys, returning None on transformers <5 where the prefix still strips and a mapping would break the load.

Apply the same family-guarded remap on the load_in_fp8 offline path and for direct FastBaseModel callers, and remap quantization llm_int8_skip_modules off the wrapper prefix after stripping.

Add a regression test that loads real VLM checkpoint weights (the prior tests only instantiated a fresh model so they missed this) and drop the bitsandbytes dependency from the quantization-config test.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Separate FP8 text-only cache and hoist the text-only guard for PR #5816

Address review of the text-only changes: (1) _offline_quantize_to_fp8 produced different artifacts for text-only vs full VLM but reused the same <name>-fp8-<mode> cache dir, so one mode could load the other's saved model; decide text-only before the cache name and add a -text-only suffix. (2) FastBaseModel.from_pretrained rewrote the VLM auto class to AutoModelForCausalLM before loading auto_config and before the family check, leaving is_vlm wrong for the fast_inference/vLLM block; hoist the family-guarded text-only decision above those checks and drop the redundant later block. (3) Wire the text-only regression test into the curated CPU pytest job so it runs in CI across the transformers matrix.

* Trim text-only code comments for PR #5816

Shorten and de-duplicate the comments added for the text-only loading work; keep the non-obvious rationale (the transformers >=5 base_model_prefix change) and drop the obvious parts. Comments only, no code changes; AST-based tests still pass on transformers 4.57.6 and 5.4.0.

* Make text-only loading opt-in via a public text_only argument for PR #5816

Rename the internal _force_text_only flag to a public text_only parameter on FastLanguageModel, FastModel and FastBaseModel (and the fp8 helper), defaulting False on all three. Text-only loading is now opt-in (text_only=True) instead of forced on by FastLanguageModel; the family guard and key remap are unchanged. Updated the AST tests for the new parameter and forwarding.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Trim text-only code comments for clarity

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>

.github/workflows/consolidated-tests-ci.yml

@@ -269,7 +269,8 @@ jobs:
             tests/saving/test_patch_saving_none_tokenizer.py \
             tests/saving/test_fix_sentencepiece_gguf_robustness.py \
             tests/utils/test_attention_masks.py \
-            tests/utils/test_trunc_normal_patch.py
+            tests/utils/test_trunc_normal_patch.py \
+            tests/python/test_fast_language_model_text_only.py
           python -m pytest --collect-only -q "$RUNNER_TEMP/unsloth-zoo/tests/"
 
       - name: import_fixes drift detectors (18 tests, HARD GATE)
@@ -333,11 +334,9 @@ jobs:
           python -m pytest -v --tb=short tests/test_callback_signature_drift.py
 
       - name: unsloth Bucket-A — CPU tests not in Repo tests (CPU)
-        # 16 tests across 5 files. They live inside tests/saving/ and
-        # tests/utils/, both of which Repo tests (CPU) excludes via --ignore
-        # because their sibling files need real GPUs / real HF weights.
-        # The five files below are pure-Python + AST/protobuf/regex tests
-        # that run cleanly on CPU. Env inherited from the job block.
+        # CPU tests across 6 files under tests/saving/, tests/utils/, tests/python/
+        # that Repo tests (CPU) --ignores. AST/protobuf/regex plus tiny CPU model
+        # loads; run cleanly here (transformers/torch installed).
         run: |
           python -m pytest -q --tb=short \
             tests/saving/test_save_shell_injection.py \
@@ -345,11 +344,12 @@ jobs:
             tests/saving/test_fix_sentencepiece_gguf_robustness.py \
             tests/utils/test_attention_masks.py \
             tests/utils/test_trunc_normal_patch.py \
+            tests/python/test_fast_language_model_text_only.py \
             --deselect 'tests/utils/test_attention_masks.py::test_run_attention_flash_varlen_receives_window_and_softcap'
           # The deselected test monkeypatches flash_attn_varlen_func, which is
           # only bound on the module when `flash_attn` is importable. flash_attn
           # requires CUDA + dev toolchain, which the CPU-only ubuntu-latest
-          # runner does not have. The other 15 Bucket-A tests pass cleanly.
+          # runner does not have. The other Bucket-A tests pass cleanly.
 
       - name: unsloth_zoo @ ${{ env.UNSLOTH_ZOO_REF }} — full pytest (CPU)
         # 106 of 111 test_* in unsloth_zoo are CPU-only. The two CUDA-skip