* Route dense NemotronH models to the transformers 5.10 tier
Dense NemotronH models (e.g. unsloth/NVIDIA-Nemotron-3-Nano-4B) describe their
layer stack with a hybrid_override_pattern that includes '-' (MLP) layers.
transformers only learned to parse that ('-' -> 'mlp' in pattern_mapping, 'mlp'
in valid_types and MIXER_TYPES) in 5.10; on 5.3/5.5 the config raises
KeyError: '-'. The model also ships auto_map remote code, so training and
inference that approve trust_remote_code load fine, but a native (TRC=False)
load such as export hits the built-in parser and fails with
'Failed to load checkpoint: -'.
Detect dense NemotronH from config.json (a '-' in hybrid_override_pattern, or
'mlp' in an expanded layers_block_type) and route it to the 5.10 tier, where the
model loads natively without remote code. Pure-MoE NemotronH configs are
unaffected and keep their existing tier.
Covers both the local config.json and the remote HF-id paths, and adds tests for
the detector and the resulting tier selection.
* Tighten _nemotron_h_needs_mlp_support docstring
* Detect dense NemotronH in nested, cached, and resolved-away configs
Three gaps could still route a dense NemotronH (MLP '-' layers) to a tier
below 5.10 and hit KeyError: '-':
- VL wrappers (e.g. NemotronH_Nano_VL_V2) keep the dense language model under
llm_config/text_config; the detector only checked the top-level model_type.
Recurse into nested language configs.
- Offline or blocked config fetches returned None for an already-downloaded
repo. Read config.json from the HF hub cache before any network.
- A local checkpoint resolves to its base before tiering, so an offline/private
base discarded the local config that revealed the dense pattern. Prefer the
higher tier of the resolved base and the original path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden NemotronH tier detection follow-ups
Address review of the nested/cached/resolved-away detection:
- The local re-check ran the full tier detector on the original path, so a bare
LoRA adapter under e.g. /runs/gemma-4-x/llama-lora could upgrade a default base
via directory-name substrings. Gate the re-check on a real local config.json so
it reads metadata, not path names.
- The HF hub cache was read before any network, so an online tier check could
serve stale config.json after the repo changed upstream. Consult the cache only
offline or after a failed fetch.
- Reading the cache imported huggingface_hub during tier detection, which runs
before a sidecar venv is activated and could pin the default-env hub into
sys.modules. Resolve the cache path with stdlib only.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim comments to be more succinct
* Select newest hub-cache snapshot by mtime and retry transient config fetches
The HF cache fallback in tier detection picked the lexicographically-first
snapshot when refs/main was absent (commit-pinned downloads), which can be an
older SHA than the Hub would load. Sort snapshots by mtime instead.
A transient online fetch failure cached the hub-cache fallback under the normal
(model_name, token) key, so a long-lived worker kept serving stale metadata even
after connectivity recovered. Return the fallback without memoizing it so the
next call retries the network.
* Harden config.json tier detection against auth failures and transient blips
- _load_config_json: a 401/403/404 from the raw Hub request is a definitive access
answer, not an outage. Return None instead of falling back to the HF hub cache, so
an unauthenticated or wrong-token request can never read another caller's cached
private metadata.
- _check_config_needs_510/550: only memoize the derived tier when the underlying
config read was definitive (local file, offline cache, or a completed fetch).
A transient fetch fallback is no longer pinned, so the tier is re-evaluated once
connectivity returns instead of staying stuck on the lower tier.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments in tier-detection auth/cache paths
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
aeb5075121