Commit graph

1 commit

Author SHA1 Message Date
Daniel Han
c520662c12
Honor an explicit sdpa or flex_attention request when flash is disabled (#6847)
* Honor an explicit sdpa or flex_attention request when flash is disabled

When flash attention is disabled for a model, the fallback selection could
downgrade a caller who explicitly passed attn_implementation='sdpa' or
'flex_attention' to a different backend, because the disable reason is
flash-specific. Keep an explicit non-flash request as-is; flash requests
still fall back as before.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments

* Gate honor-explicit attention on provenance and flex support

Only honor an explicit non-flash attention request when it comes from the
caller argument, not from a config value the loaders synthesize (the language
path seeds attn_implementation=sdpa). Honor explicit flex_attention only when
supports_flex_attention is True so excluded/broken configs (e.g. gpt_oss) fall
back instead of selecting a known-broken backend. Explicit sdpa stays honored.

* Honor explicit sdpa through the resolver guard

* Keep SDPA exclusions when honoring an explicit sdpa request

An explicit attn_implementation="sdpa" was re-enabling sdpa for models in
_SDPA_EXCLUDED_MODELS (e.g. gpt_oss) where sdpa is known-broken: the helper
honored the request and the resolver's final not-supports_sdpa guard skipped
the eager downgrade for any explicit request. Honor an explicit sdpa only when
the model is not sdpa-excluded, mirroring the flex guard that already falls
back for _FLEX_EXCLUDED_MODELS via supports_flex_attention. Conservative
supports_sdpa=False (large head dim / attention-sink models) still honors an
explicit sdpa; a synthesized/default sdpa still downgrades to eager.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Honor DISABLE_SDPA_MODEL_NAMES when honoring explicit sdpa

The honor-explicit-sdpa guard only skipped the sdpa->eager downgrade for models
in _SDPA_EXCLUDED_MODELS (gpt_oss). Gemma3/Gemma3Text disable SDPA through the
loader's DISABLE_SDPA_MODEL_NAMES (their bundled SDPA modules are wrong), so an
explicit sdpa request bypassed the downgrade and re-enabled a known-wrong path.

Extend _is_sdpa_excluded to also treat DISABLE_SDPA_MODEL_NAMES membership as
excluded, replicating the loader's trailing-comma substring match so gemma3 and
gemma3_text match but gemma3n does not. Move the constant into _utils.py (single
source of truth, re-exported from loader.py) to avoid a loader -> _utils cycle.
Conservative supports_sdpa=False models not in either list still honor explicit
sdpa.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-06 05:45:45 -07:00