docker: mirror soname symlinks into llama.cpp build/bin, assert the relinked quantizer executes
The build/bin hardlink mirror skipped symlinks, so the soname links (libllama-common.so.0 and friends) never reached build/bin. Studio's setup.sh relinks the root llama-quantize to build/bin/llama-quantize, whose RUNPATH is $ORIGIN, so the loader failed with libllama-common.so.0 not found and GGUF export from Studio died with No working quantizer found, then hit the interactive source-build prompt in a non-TTY export subprocess (EOFError). Mirror same-directory soname symlinks into build/bin and extend the bake sanity check to execute llama-quantize from both the install root and build/bin. Dockerfile.studio now also runs the studio-visible quantizer after install.sh so a regression fails the image build instead of runtime exports.
This commit is contained in:
parent
6f6b6389e1
commit
8242b73c88
2 changed files with 35 additions and 11 deletions
|
|
@ -107,6 +107,10 @@ RUN set -eux \
|
|||
# wheels with no sm_100/sm_120 kernels). metadata check only: importing
|
||||
# torch needs native libs, which QEMU arm64 builds cannot load.
|
||||
&& "${UNSLOTH_STUDIO_HOME}/unsloth_studio/bin/python" -c "from importlib.metadata import version; v = version('torch'); assert v.endswith('+${TORCH_FAMILY}'), 'Studio venv torch ' + v + ' does not match ${TORCH_FAMILY}'; print('Studio venv torch', v)" \
|
||||
# setup.sh may relink the root llama-quantize into build/bin; prove the
|
||||
# relinked quantizer still resolves its libraries, or GGUF export breaks
|
||||
# at runtime with "No working quantizer found".
|
||||
&& "${UNSLOTH_STUDIO_HOME}/llama.cpp/llama-quantize" --version \
|
||||
&& rm -rf "${UNSLOTH_STUDIO_HOME}/src/.git" /root/.cache \
|
||||
&& if [ "${TARGETARCH:-amd64}" = "arm64" ]; then \
|
||||
for NVRTC_DIR in "${UNSLOTH_STUDIO_HOME}"/unsloth_studio/lib/python*/site-packages/nvidia/cuda_nvrtc/lib; do \
|
||||
|
|
|
|||
|
|
@ -140,21 +140,41 @@ def main() -> None:
|
|||
os.link(source, os.path.join(build_bin, entry))
|
||||
except OSError:
|
||||
shutil.copy2(source, os.path.join(build_bin, entry))
|
||||
elif os.path.islink(source):
|
||||
# Mirror same-directory soname symlinks (libllama.so.0 -> ...).
|
||||
# Without these, a binary relinked into build/bin fails $ORIGIN
|
||||
# resolution: the loader wants the soname, not the real file.
|
||||
target = os.readlink(source)
|
||||
dest = os.path.join(build_bin, entry)
|
||||
if "/" not in target and not os.path.lexists(dest):
|
||||
os.symlink(target, dest)
|
||||
|
||||
# Sanity: the server binary must execute on a GPU-less host (the CUDA
|
||||
# backend is a dlopen'd plugin, so --version works anywhere).
|
||||
out = subprocess.run(
|
||||
[os.path.join(install_dir, "llama-server"), "--version"],
|
||||
capture_output = True,
|
||||
text = True,
|
||||
timeout = 120,
|
||||
# backend is a dlopen'd plugin, so --version works anywhere). Check the
|
||||
# quantizer from BOTH roots: Studio's setup.sh relinks the root
|
||||
# llama-quantize to build/bin/llama-quantize, so the build/bin copy must
|
||||
# resolve its libraries standalone.
|
||||
checks = (
|
||||
# llama-quantize has no --version; a healthy run prints usage with
|
||||
# rc 0, while a loader failure prints to stderr with rc 127.
|
||||
(os.path.join(install_dir, "llama-server"), "version"),
|
||||
(os.path.join(install_dir, "llama-quantize"), "usage"),
|
||||
(os.path.join(build_bin, "llama-quantize"), "usage"),
|
||||
)
|
||||
banner = (out.stdout + out.stderr).strip()
|
||||
print(banner.splitlines()[0] if banner else "(no version banner)")
|
||||
if "version" not in banner:
|
||||
raise SystemExit(
|
||||
f"FAIL: llama-server --version did not report a version: rc={out.returncode}"
|
||||
for binary, expect in checks:
|
||||
out = subprocess.run(
|
||||
[binary, "--version"],
|
||||
capture_output = True,
|
||||
text = True,
|
||||
timeout = 120,
|
||||
)
|
||||
banner = (out.stdout + out.stderr).strip()
|
||||
print(os.path.relpath(binary, install_dir), "->",
|
||||
banner.splitlines()[0] if banner else "(no output)")
|
||||
if expect not in banner:
|
||||
raise SystemExit(
|
||||
f"FAIL: {binary} did not print '{expect}': rc={out.returncode}\n{banner[:400]}"
|
||||
)
|
||||
for required in (
|
||||
"llama-quantize",
|
||||
"convert_hf_to_gguf.py",
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue