67f0ab592e
When llama-server is built with shared libs (setup.sh default), it needs libcudart.so.12 and other CUDA runtime libs. Add /usr/local/cuda/lib64 and targets path to LD_LIBRARY_PATH so the server starts correctly even when CUDA isn't on the system path. |
||
|---|---|---|
| .. | ||
| assets | ||
| auth | ||
| core | ||
| loggers | ||
| models | ||
| plugins | ||
| requirements | ||
| routes | ||
| state | ||
| tests | ||
| utils | ||
| __init__.py | ||
| colab.py | ||
| main.py | ||
| run.py | ||