* fix: keep offline GGUF export off the Hub for VLM tokenizers (#7481)
Resolve cached snapshot directories before loading PreTrainedTokenizerFast
during VLM processor fallback so transformers does not call is_base_mistral()
-> model_info() when HF_HUB_OFFLINE is set. Also probe the local cache in
_has_tokenizer_model instead of model_info when offline.
Fixesunslothai/unsloth#7481
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* test: add real-cache offline GGUF integration checks for #7481
Download unsloth/gemma-3-270m-it-bnb-4bit (~430MB) and verify offline
snapshot resolution and tokenizer load with network blocked. Full unsloth
import tests remain GPU-gated.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: address Codex review on offline GGUF tokenizer paths (#7481)
- Only rewrite Hub repo ids to cached snapshot dirs when offline
- Copy tokenizer.model from cache offline in preserve_sentencepiece
- Do not cache negative offline tokenizer.model probe results
- Add regression tests for all three review items
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: probe HF cache before model_info for local-only GGUF saves (#7481)
Always resolve tokenizer.model from the local Hub cache before calling
model_info, and skip Hub metadata when the tokenizer was loaded with
local_files_only or offline env vars. Fixes Codex review on PR #7482.
* Fix lint blocker, false-green tests and offline defaults for PR #7482
Drop the two unused _env_says_offline imports that fail the Source lint
import-hoist check.
test_has_tokenizer_model_offline_skips_model_info and its local_files_only
twin set model_info.side_effect = AssertionError, but _has_tokenizer_model
wraps that call in "except Exception: return False", so the AssertionError
was swallowed and both passed on the merge base with the fix absent. Assert
model_info.call_count == 0 instead; both now fail on the base with
assert 1 == 0.
The real-cache integration tests called hf_hub_download and
PreTrainedTokenizerFast directly, so they exercised plain huggingface_hub and
passed identically on both trees. Route them through the resolver this PR
adds, and gate the file at module level since importing unsloth needs a GPU
host either way.
_resolve_hub_repo_local_dir and _resolve_hub_repo_cached_file defaulted to
local_files_only = False, so a helper named "resolve local dir" would
download with backoff retries when called without the flag. Every caller
already passes it explicitly, so default it closed.
Use tempfile.gettempdir() rather than a hardcoded /tmp, which silently
skipped both files on Windows, the platform in the bug report. Patch
socket.socket connect rather than replacing the class, which broke
isinstance checks.
Wire the unit tests into the Bucket-A CI list; Repo tests (CPU) ignores
tests/saving, so none of these ran anywhere.
* docs: note transformers 4.57.2-5.5.4 window for local tokenizer resolve
Name the version range where from_pretrained still probes model_info under
local_files_only, and point at the 5.6.0 upstream fix so the helper can be
removed once the supported floor moves past it.
* fix: enable real-cache suite in offline GGUF integration runner
Pass UNSLOTH_INTEGRATION_IMPORT=1 into the pytest subprocess so the
documented runner actually executes the real-cache tests instead of
reporting success after only the fake-cache unit file runs.
* docs: note integration runner enables UNSLOTH_INTEGRATION_IMPORT
Document that the runner sets the gate itself and still needs a host
that can import unsloth.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep an explicit local_files_only load local-only at save time
transformers takes local_files_only as an explicit from_pretrained parameter,
so it never lands in tokenizer.init_kwargs, and _offline_aware_load restores
HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE as soon as the load window closes. A VLM
loaded with local_files_only = True but no offline env var therefore came back
with the Hub repo id in name_or_path and nothing recording the request, so
_tokenizer_wants_local_only returned False on the later save and
_has_tokenizer_model fell through to HfApi.model_info - and then
_preserve_sentencepiece_tokenizer_assets fetched tokenizer.model from the Hub
with local_files_only = False. On a disconnected host that is a network wait
before the export gives up.
Stamp the load's local-only mode onto the returned processor and its tokenizer
inside the forced-offline window, and honour that stamp in
_tokenizer_wants_local_only, so the save path inherits the load's contract.
Verified against a real hub-cache layout whose snapshot has tokenizer metadata
but no tokenizer.model: before, one model_info call plus an hf_hub_download with
local_files_only = False; after, zero model_info calls and cache probes only.
Two tests added to tests/saving/test_offline_gguf_vlm_tokenizer_7481.py; both
fail with the loader_utils hunk reverted and pass with it in place.
* Carry the load's cache_dir through to saving for PR #7482
The local-only stamp added in e7b7400de preserved only the boolean. Saving
still derived its cache from HF_HUB_CACHE or HF_HOME, which does not see a
caller-supplied cache_dir, and FastBaseModel.from_pretrained threads one all
the way down. So a local_files_only load against a custom cache missed on the
probe, and the stamp then stopped the Hub fallback that used to cover it, and
tokenizer.model was silently left out of the GGUF staging directory.
Stamp the cache_dir alongside the local-only marker and prefer it at both
sites in save.py that derive one from the environment. Reverting save.py
alone, with the helper still present, fails the new test on behaviour.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Merge main and drop an unused import for PR #7482
Brings the branch up to date with main, which clears the stale Source lint
blocker inherited from #7476 by taking studio/backend/utils/hardware/__init__.py
out of this PR's changed-file set.
pytest was imported in the new test file and never used, which the
import-hoist check flags in its own right.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
8b9ee5facb