* fix: add XPU device support and update hardcoded CUDA selections
* fix: add XPU device support for pytest CUDA skipped tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix device handling for PR #7401
- perplexity_eval.py: use DEVICE_TYPE_TORCH, not DEVICE_TYPE. The latter can
be "hip" or "mlx", which .to() rejects, so this regressed ROCm.
- test_batched_leftpad_generation_gpu.py: XPU diverges here today, so mark it
non-strict xfail on XPU instead of reverting to a CUDA-only guard. Keeps the
real XPU gap visible and turns green once it is fixed.
- Guard torch.xpu.is_available() with hasattr, matching device_type.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Re-enable the flash varlen attention test in CI for PR #7401
attention_dispatch.py now predefines flash_attn_func / flash_attn_varlen_func
as None, so test_run_attention_flash_varlen_receives_window_and_softcap no
longer needs flash_attn importable to be monkeypatched. Verified on a runner
shaped like the CPU-only one: the test fails against main's attention_dispatch
and passes at this head, so the deselect is now dead weight.
* Tighten comments for PR #7401
Drop the hasattr rationale: torch.xpu has existed since torch 2.3 and the
dependency floor is 2.4, so no supported build predates the namespace. The
guard stays as cheap defence, but the comment claimed something untrue.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Handle odd shapes and non-float scales in FP8BlockQuantLinear
Small fp8 checkpoints (e.g. tiny test models) break the block-quantized
linear in three ways: weight scales stored in a float8 dtype such as
float8_e8m0fnu have no triton dtype mapping; activations whose hidden dim is
not a multiple of the activation quant block fail act_quant's divisibility
assert; and weights whose dims are not multiples of the weight block cannot
be tiled by the triton dequant kernel.
Cast non-float scales to float32 on entry, and when the hidden dim does not
divide into the activation block, dequantize the weight and run a plain
matmul instead of the fp8 block matmul. The dequant goes through a new
shape-safe helper that falls back to a torch-native scale expansion when the
weight does not tile evenly; backward uses the same helper so the gradient
path works for every shape the forward accepts. Full-size checkpoints are
unaffected.
* Add tiny / e8m0 fp8 block-quant regression test
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix FP8 block-quant fallback: real block size in dequant and scalar-scale fast path
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Route rectangular fp8 blocks through torch dequant and keep block_size across e8m0 upcast
The triton weight_dequant kernel uses one BLOCK_SIZE for both axes, so
rectangular blocks (block_size[0] != block_size[1]) mis-index the column
scale and corrupt grad_X. Route those through the torch scale expansion,
which handles each dimension independently, and keep the triton path for
square blocks only.
Also preserve a block_size attribute carried on the scale tensor across the
e8m0 -> float32 upcast so the later lookup no longer falls back to [128, 128].
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
