docker: mirror the llama.cpp bake into build/bin so Studio setup reuses it
Studio's setup.sh provisioning runs install_llama_prebuilt.py, whose host-probing cannot succeed inside an image build, so it fell back to a CPU-only llama.cpp source build layered over the baked CUDA bundle. setup.sh skips that fallback when build/bin/llama-server and build/bin/llama-quantize are executable, so hardlink the installed bundle into build/bin: zero extra bytes, $ORIGIN rpath still resolves, and no symlink cycle when setup.sh later relinks the root quantizer to build/bin/llama-quantize.
This commit is contained in:
parent
d431f3cf42
commit
6fd1220ba0
1 changed files with 19 additions and 0 deletions
|
|
@ -122,6 +122,25 @@ def main() -> None:
|
|||
if os.path.isdir(conversion):
|
||||
shutil.copytree(conversion, os.path.join(install_dir, "conversion"), dirs_exist_ok = True)
|
||||
|
||||
# Mirror the install into build/bin/ via hardlinks (zero extra bytes).
|
||||
# Studio's setup.sh treats an executable build/bin/llama-server +
|
||||
# build/bin/llama-quantize as a complete local build and skips its
|
||||
# source-build fallback -- which would otherwise fire inside the image
|
||||
# build, where the host-probing prebuilt updater cannot succeed, and
|
||||
# compile a CPU-only llama.cpp over the baked CUDA bundle. Hardlinks
|
||||
# (not symlinks) keep $ORIGIN rpath resolution working from build/bin
|
||||
# and avoid a cycle when setup.sh later relinks the root quantizer to
|
||||
# build/bin/llama-quantize.
|
||||
build_bin = os.path.join(install_dir, "build", "bin")
|
||||
os.makedirs(build_bin, exist_ok = True)
|
||||
for entry in os.listdir(install_dir):
|
||||
source = os.path.join(install_dir, entry)
|
||||
if os.path.isfile(source) and not os.path.islink(source):
|
||||
try:
|
||||
os.link(source, os.path.join(build_bin, entry))
|
||||
except OSError:
|
||||
shutil.copy2(source, os.path.join(build_bin, entry))
|
||||
|
||||
# Sanity: the server binary must execute on a GPU-less host (the CUDA
|
||||
# backend is a dlopen'd plugin, so --version works anywhere).
|
||||
out = subprocess.run(
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue