* Studio: add torch's pip nvidia DLL dirs to PATH on Windows
Studio's install_python_stack bundles torch with matching CUDA
wheels (nvidia-cuda-runtime-cu13, nvidia-cublas-cu13, etc.) which
ship cudart64_X.dll, cublas64_X.dll, and cublasLt64_X.dll under
the prefix's Lib/site-packages/nvidia/<pkg>/(bin|Library/bin)/
tree. The Linux runtime env block in start_llama_server already
pulls the equivalent nvidia/cu*/lib paths into LD_LIBRARY_PATH,
but the Windows block did not do this, so the prebuilt
llama-server.exe could not resolve cudart64_X.dll at runtime
unless the user had a matching system CUDA toolkit on PATH. That
is the root cause of the Windows reports in
unslothai/unsloth#5106 ("GPU detected but model loaded entirely
on RAM/CPU"), and matches Roland's repeated workaround in that
issue: install matching CUDA toolkit version.
Brings the Windows env block in line with the Linux pattern:
* New LlamaCppBackend._windows_pip_nvidia_dll_dirs resolver
globs <prefix>/Lib/site-packages/nvidia/<pkg>/bin and
<prefix>/Lib/site-packages/nvidia/<pkg>/Library/bin. Both
layouts are seen in the wild across cuda_runtime / cublas /
cudnn / nvjitlink wheels.
* The Windows env block now extends path_dirs with the
resolver's output before falling back to CUDA_PATH/bin, so
pip-installed wheels are the canonical source (mirroring the
Linux LD_LIBRARY_PATH ordering). System CUDA toolkit remains a
valid fallback.
Tests: 7 new cases in
studio/backend/tests/test_llama_cpp_windows_nvidia_path.py:
* empty resolver when no nvidia wheels installed
* nvidia/<pkg>/bin layout resolved
* nvidia/<pkg>/Library/bin layout resolved
* mixed bin and Library/bin layouts both resolved
* unrelated site-packages contents not walked
* non-directory entries skipped
* missing prefix does not raise
110 backend tests pass. No regressions.
Refs #5106
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: also scan torch/lib in Windows pip nvidia DLL resolver
PyTorch's Windows CUDA wheels frequently bundle cudart64_X.dll and
cublas64_X.dll directly under Lib/site-packages/torch/lib/ instead of
shipping separate nvidia-cuda-runtime-cuXX / nvidia-cublas-cuXX wheels.
On those installs _windows_pip_nvidia_dll_dirs previously returned
nothing useful, and llama-server.exe fell back to needing a system CUDA
toolkit on PATH -- the original #5106 failure mode.
The install-side equivalent python_runtime_dirs in
install_llama_prebuilt.py already treats torch/lib as a Python runtime
DLL source for the same reason. Bring the runtime resolver in parity
so torch-bundled-CUDA installs find their cudart at llama-server start.
Updates the existing test that codified the bug (asserted torch/lib was
excluded), and adds three new cases: pickup, combined-with-nvidia, and
the must-be-a-directory guard.
* Studio: cover cu13 bin/x86_64 layout in Windows DLL resolver
Three follow-ups from a 12-reviewer batch over c1c8a074 (PR #5324):
1. The current nvidia-cuda-runtime (unsuffixed) 13.2.75 and
nvidia-cublas 13.4.0.1 Windows wheels on PyPI ship under
nvidia/cu13/bin/x86_64/cudart64_13.dll etc, not under
nvidia/PKG/bin/. The previous resolver matched only one
directory level past nvidia/PKG/ and silently missed the
actual cu13 DLL location, leaving CUDA 13 users on the same
failure mode as before #5106. Verified against:
pip download nvidia-cuda-runtime --platform win_amd64
which produces nvidia/cu13/bin/x86_64/cudart64_13.dll.
2. glob.glob over sys.prefix interprets [ and ] as a
character class. Valid Windows usernames / install paths can
contain those characters (for example C:\Users\alice[work]\studio),
so the previous resolver silently returned an empty list for such
prefixes even when DLL dirs were present.
3. The resolver only ever returned nvidia/PKG/bin -- if both
bin and bin/x86_64 exist (current wheels do), Windows
DLL search should land on the arch-specific subdir first so the
explicit cudart64_X.dll location wins.
Rewritten as a pathlib.Path.iterdir walk to fix all three:
no glob escaping needed, arch-specific subdirs added explicitly,
and ordering puts bin/x86_64 before bin. Conda-style
Library/bin/x86_64 and Library/bin/x64 are also covered for
parity. A seen set dedupes when wheels happen to expose the
same directory through multiple layouts.
New tests:
- test_picks_up_cu13_bin_x86_64_layout (the actual real-world cu13 case)
- test_picks_up_bin_x64_layout
- test_mixed_cu12_and_cu13_layouts
- test_glob_meta_in_prefix_is_safe (bracket repro)
- test_arch_subdir_listed_before_parent_bin (ordering)
Verified empirically against PyPI:
nvidia-cuda-runtime 13.2.75 -> nvidia/cu13/bin/x86_64/cudart64_13.dll
nvidia-cublas 13.4.0.1 -> nvidia/cu13/bin/x86_64/cublas64_13.dll
nvidia/cu13/bin/x86_64/cublasLt64_13.dll
nvidia-cudnn-cu13 9.22.0.52 -> nvidia/cudnn/bin/cudnn64_9.dll (already covered)
Refs #5106
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
379f5a5aa6