torchao has no working Windows ROCm build. transformers.quantizers imports it,
and it loads torch's c10d distributed backend at module level, which the AMD
Windows wheels omit (no RCCL). The import aborts, transformers can no longer
expose PreTrainedModel, and the sentence-transformers embedder silently falls
back to the llama-server GGUF embedder. Linux ROCm and NVIDIA are unaffected
(the c10d ops are present / torchao is real there).
The training and export workers already install the shared torchao stub before
importing transformers, but the RAG embedder runs in the main backend process,
which never did. Two fixes, both no-ops off Windows ROCm:
- embeddings.py: install_torchao_windows_rocm_stub() before the first
sentence-transformers import, so an already-installed torchao is neutralized
(fixes existing venvs).
- install_python_stack.py: stop installing torchao on Windows ROCm; it can only
crash on import there, so new venvs never ship it.
Add tests covering the embedder stub call and the install skip.
* studio: select torchao version from the installed torch
The Studio installer pins CUDA torch to torch>=2.4,<2.11 and its driver
ladder selects the cu130 wheel index on recent NVIDIA drivers, so pip
resolves torch 2.10.0. overrides.txt hard-pinned torchao==0.14.0, whose
C++ extensions are built against torch 2.9.0, so torchao skipped its cpp
kernels ("Skipping import of cpp extensions due to incompatible torch
version 2.10.0+cu130 for torchao version 0.14.0") and fell back to the
slow Python path. Every CUDA index now tops out at torch 2.10.0, so this
hit most modern installs, not just cu130.
Pick the torchao version matching the torch actually installed in the
venv (table: pytorch/ao#2919): torch 2.10.x -> torchao 0.16.0, 2.11.x ->
torchao 0.17.0, otherwise the previous 0.14.0 (so torch <=2.9 is
unchanged). The installer reads torch.__version__ from the venv via a
cross-platform sys.executable probe (probe_torch_wheel_env is Linux-only)
and passes the computed spec positionally to the existing force-reinstall
override step; overrides.txt becomes a pointer to that logic. torchao's
Python API (Float8Tensor, used by unsloth/kernels/utils.py) imports
cleanly on 0.16.0/0.17.0, verified against torch 2.9.1.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: address review on torchao selection
- Clean the torch minor of pre-release/dev suffixes before parsing
(e.g. '2.10rc1' -> minor 10), matching wheel_utils.probe_torch_wheel_env.
- Pass _windows_hidden_subprocess_kwargs() to the torch-version probe so
it does not flash a console window on Windows (no-op elsewhere).
- Use _safe_print for the selection log line, consistent with the file's
other status output (safe on non-UTF-8 consoles).
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
