When llama-server is built with shared libs (setup.sh default), it needs libcudart.so.12 and other CUDA runtime libs. Add /usr/local/cuda/lib64 and targets path to LD_LIBRARY_PATH so the server starts correctly even when CUDA isn't on the system path. |
||
|---|---|---|
| .. | ||
| backend | ||
| frontend | ||
| __init__.py | ||
| install_python_stack.py | ||
| LICENSE.AGPL-3.0 | ||
| setup.bat | ||
| setup.ps1 | ||
| setup.sh | ||
| Unsloth_Studio_Colab.ipynb | ||